Works well. I really like the badge on the icon that indicates whether the page is blocked by robots.txt or not.
Robots.txt Tester
- 120 users
- 5.001 rating
- #5,912in Developer Tools
Check robots.txt of any site with Robots.txt Tester: highlighted syntax, error warnings, allow/disallow verdicts per user-agent.
by Robots Dev · 2 listings
- Users
- 120
- #5,912 of 31,192 in category
- Rating
- 5.00
- 1 rating · median 4.63
- Version
- 1.0.2
- 617 KiB · Manifest V3
- Access
- Medium54/100
- 2 permissions + every website
Description
As written by the publisher on the storeRobots.txt Tester shows a robots.txt file of any website you have open: click its toolbar icon and get every directive back with color highlighting, syntax warnings, URL testing — no third-party websites, no copy-paste.
One click fetches this file from your current domain, renders User-agent, Allow, Disallow lines in color, flags syntax problems right away: unknown fields, stray characters, malformed rules. As a robots.txt tester tool it audits only — nothing gets overwritten, nothing gets edited.
- How to use it:
- Open any website in a Chrome tab.
- Click Robots.txt Tester in your Chrome toolbar.
- Read highlighted directives plus a warnings list.
- Paste a URL, pick a user-agent, get a verdict: allowed or blocked.
- 🧪 Test URLs without leaving a page
- One question matters most: is this URL allowed or blocked? This popup answers it directly:
- Test robots.txt rules against any URL of your current domain.
- See a clear verdict with its matching rule quoted underneath.
- Switch user-agents to compare how different crawlers behave.
- Validate syntax while auditing, not after.
- Works over https, http, localhost — useful during local development.
- Everything runs in one popup — this robots.txt checker needs no signup, no extra tabs.
🤖 Check robots.txt for every important bot
Search engines, Bing, AI crawlers — each follows its own user-agent. Pick one from this list or type your own:
🔹 Googlebot — make sure Google reaches pages that matter.
🔹 Bingbot — verify Bing access before blaming indexing.
🔹 GPTBot — see whether AI crawlers read your content, or confirm that you block AI crawlers on purpose.
🔹 Custom agent — type any user-agent string, run one test for it.
- 📌 How server responses are interpreted
- HTTP 200 — rules parsed, applied as written.
- HTTP 404 — no such file exists, so everything is allowed.
- Server errors (5xx) — treated as full disallow, strictest case.
- Redirects — followed to a final URL, shown inside this popup.
- Google treats 4xx as fully allowed, 5xx as fully disallowed — this extension applies the same logic automatically.
💡 Quick answers
Five questions SEO teams ask most:
❓ How to test robots.txt without online tools?
Open a site, click this icon, paste a URL you care about. Validation runs locally in your browser.
❓ Can I test my robots.txt before publishing it?
Yes — point this robots.txt tester at a staging domain, compare verdicts bot by bot before deploying.
❓ Do I need a robots.txt generator?
Not for audits. This robots.txt validator reads, explains a live file instead of creating one, so you see exactly what crawlers see.
❓ Where do I find a robots.txt example?
Every website you visit is one: open a tab, click this icon, study real robots txt format with real directives.
❓ What about subdomains?
Each subdomain keeps its own file: switch tabs between example.com, shop.example.com, blog.example.com to compare.
🌐 Why it matters for search
Crawling, indexing start with this small file. One misplaced disallow, one missing sitemap line, one greedy wildcard can hide important pages from search engines. This robots.txt tester was built for exactly that moment: catch problems before release, not after rankings drop. Crawl budget matters on large sites too — when crawlers waste requests on blocked URLs, fresh content gets discovered slower. A clean file plus a sitemap reference keeps crawling focused. That is what a proper seo workflow looks like.
- 🔍 Mistakes that hide pages from search
- Disallow inside a wrong User-agent group blocks one bot only.
- Disallow: / without an Allow exception shuts down an entire site.
- Greedy wildcards like /*.php$ drop more URLs than intended.
- Missing sitemap lines slow down discovery of new pages.
- Blocking css or js files breaks rendering for search engines.
- With this extension you spot each mistake in seconds — inside warnings or through a quick URL check.
🔒 Private by design
Everything parses locally, your data never leaves your browser. No accounts, no analytics, no third-party requests — just one fetch per domain you check.
🎯 Who it is for
SEO specialists auditing client sites, developers shipping deploys, site owners curious what googlebot or bingbot can reach. Use it to compare production with staging before a deploy, to explain a blocking issue to a client, to double-check that a redesign did not lock out crawlers. Even if you typed robot.txt by mistake, this robots txt checker is what you were looking for — one click, no setup.
Install Robots.txt Tester, turn every tab into an online robots.txt tester. Check, test, understand crawl rules, sitemap lines, wildcards in seconds — straight from your Chrome toolbar.
Ratings and reviews
Split of the 1 written reviews we hold1 written review we hold
Track Robots.txt Tester on Radar
This page shows the listing as it is today. Add it to Radar and get one email when it changes.
- Users and rating
- Store text and screenshots
- Permissions it asks for

Robots.txt Tester, in plain words
What it is, what to use instead, what it can access and how to fix it.
Robots.txt Tester at a glance
Robots.txt Tester is a Developer Tools extension by Robots Dev with 120 users. Chrome Web Store users rate it 5.00 out of 5 from 1 rating, which is still a small sample. By users it sits in the top 25% of Developer Tools extensions.
- By users
- #5,912of 31,192 in Developer Tools
- Rating
- 5.00category median 4.63
- Access
- Medium54 of 100
Robots.txt Tester alternatives
Closest by what they do · users as the store shows them · bars on a log scaleIs Robots.txt Tester safe?
Robots.txt Tester asks for 2 permissions and access to every website. 2 of them give wide access.
Permissions show what it could do, not what it does. See the full permission list.
<all_urls>Read and change data on every websitetabsSee the address and title of every open tab
Every permission Robots.txt Tester asks for
2 permissions + every website · 2 sensitive · Medium access overallSites
- Read and change data on every website
http://*/*
Browser permissions · 2
- See the address and title of every open tab
tabs - Save its own settings
storage
Sensitive marks access that reaches your data or other sites. The access level is a 0 to 100 score of all of it: under 25 is Low, 60 and up is High.
Robots.txt Tester not working?
What recent reviews say
Store reviews from the last 90 daysCommon fixes
Try them in this order- 1Update Chrome and the extension
- 2Turn it off and on in chrome://extensions
- 3Reload the page
- 4Turn other extensions off in case two clash
Still stuck? Contact the maker or try an alternative.
Know when a listing changes
Track any extension on Radar: users, rating, permissions and store text, with one email when something moves.