Errors in SEO: «Blocked by X-Robots-Tag»

Errors List

1. 3xx Redirected Pages
2. 4xx Error Pages: Client Error
3. 5xx Error Pages: Server Error
4. Bad AMP HTML Format
5. Bad Base Tag Format
6. Blocked by Meta Robots
7. Blocked by Robots.txt
8. Blocked by X-Robots-Tag
9. Broken Images
10. Broken Pages
11. Broken Redirect
12. Canonical Chain
13. Canonical URL Contains Another Host
14. Canonicalized Pages
15. Compliant AMP Pages
16. Compliant URLs with Query
17. Contains Flash
18. Contains Lorem Ipsum
19. Duplicate Descriptions
20. Duplicate H1
21. Duplicate Pages
22. Duplicate Text
23. Duplicate Titles
24. Emojis or Special Characters in Description
25. Emojis or Special Characters in Title
26. Endless Redirect
27. External Dofollow Links
28. External Nofollow Links
29. External Redirect
30. GA: Compliant Pages w/o Traffic
31. GA: Max Bounce Rate
32. GA: Non-Compliant Pages with Traffic
33. GSC: Compliant Pages w/o Clicks
34. GSC: Compliant Pages w/o Impressions
35. GSC: Non-Compliant Pages with Impressions
36. Hreflang: Duplicate Alternate URLs
37. Hreflang: Duplicate Language Codes
38. Hreflang: Inconsistent Language Code in Confirmation Links
39. Hreflang: Incorrect Language Codes
40. Hreflang: Links to Non-Compliant URLs
41. Hreflang: Missing Alternate URLs
42. Hreflang: Missing Confirmation Links
43. Hreflang: Missing Self-Reference
44. Hreflang: Relative Links
45. HTTPS → HTTP Hyperlinks
46. HTTPS → HTTP Redirects
47. Hyperlinks with Empty Anchor Text
48. Identical Canonical URLs
49. Images Without Alt Attributes
50. Images Without Title Attributes
51. Internal Nofollow Links
52. Links to Localhost
53. Links with Bad URL Format
54. Long Server Response Time
55. Max Content Size
56. Max Description Length
57. Max External Links
58. Max H1 Length
59. Max HTML Size
60. Max Image Size
61. Max Internal Links
62. Max Redirections
63. Max Title Length
64. Max URL Length
65. Min Content Size
66. Min Text/HTML Ratio
67. Missing Internal Links
68. Missing or Empty Description
69. Missing or Empty H1
70. Missing or Empty Robots.txt File
71. Missing or Empty Title
72. Missing Structured Data
73. Mixed Content
74. Multiple Descriptions
75. Multiple H1
76. Multiple Titles
77. Nofollowed by Meta Robots
78. Nofollowed by X-Robots-Tag
79. Non-Compliant Canonical URL
80. Non-HTTPS Protocol
81. PageRank: Dead End
82. PageRank: Missing Outgoing Links
83. PageRank: Orphan
84. PageRank: Redirect
85. Percent-Encoded URLs
86. Query Contains Question Mark
87. Redirect Blocked by Robots.txt
88. Redirect Chain
89. Redirects with Bad URL Format
90. Refresh Redirected
91. Same Title and Description
92. Same Title and H1
93. Self-Referencing Hyperlinks
94. Short Description
95. Short H1
96. Short Title
97. Spelling Mistakes
98. URLs with Unwanted Special Characters
99. URLs with Uppercase Characters
100. URLs with UTM Parameters
101. URLs with Wrong Hyphenation

Importance: Medium

Error «Blocked by X-Robots-Tag» Description

Indicates URLs that contain the 'noindex' or 'none' directive in the X-Robots-Tag HTTP header.

The Importance of the Problem

Instructions in the X-Robots-Tag header help to disallow search engines from showing website pages in search results, but they are not supported by all search engines.

The X-Robots-Tag is useful to disallow indexing of non-HTML documents because it's located in the server response header. Nevertheless, it can also be used for HTML pages.

It's not recommended to use both Meta Robots and X-Robots-Tag for HTML pages. If different directives appear in these settings by any mistake, important pages may lose traffic.

For these directives to be followed by search robots, the URL must not be blocked by a robots.txt file.

How to Fix Error

Make sure that search engines are allowed to show important pages in search, while unimportant are the only ones that are disallowed.

If both Meta Robots and X-Robots-Tag are used simultaneously for HTML pages, make sure they match or better leave only one of them: either the meta tag or HTTP header.