Catalog › Robots.txt › robots.txt inaccessible
robots.txt inaccessible
robots.txt could not be fetched. Crawlers may treat the entire site as disallowed.
| Check ID | ROBOTS_TXT_INACCESSIBLE |
|---|---|
| Category | Robots.txt |
| Severity | Error |
| Plans | Available on every plan |
How to fix it
Make /robots.txt return a 200. If you have no rules, serve an empty allow-all file.
Catch it on your site
Wrendex catches robots.txt inaccessible on every audit. No credit card, all 140+ checks, full PDF report.