Should I block or allow GPTBot on my website?
Allow it, in almost every case. GPTBot is OpenAI’s crawler, and blocking it in robots.txt means ChatGPT cannot cite your website at all. The same logic applies to PerplexityBot, ClaudeBot, and Google-Extended, which covers Gemini and AI Overviews. Blocking a crawler removes you from that platform’s answers entirely.
Updated
Why does gptbot matter for a contractor?
Some website platforms and security plugins block AI crawlers by default, so contractors are frequently blocking ChatGPT without ever choosing to. This is the first thing worth checking, because every other AI visibility effort is wasted if the crawler cannot reach the page.
What does it cost to ignore it?
A blocked crawler makes your business permanently unciteable on that platform regardless of how good the content is — the most expensive possible outcome from a one-line file.
Which engine fixes this?
GPTBot is addressed inside The Stability Engine.
See what The Stability Engine includesCommon questions
Is there any AI crawler worth blocking?
CCBot, the Common Crawl bot, is the usual candidate. It feeds training datasets rather than live search, so blocking it limits use of your content for model training while leaving the search crawlers — GPTBot, PerplexityBot, ClaudeBot, Google-Extended — free to cite you. That is a reasonable middle ground if training use concerns you.
How do I check whether I am blocking AI crawlers?
Open yourdomain.com/robots.txt and look for Disallow rules naming GPTBot, ChatGPT-User, PerplexityBot, ClaudeBot, anthropic-ai, or Google-Extended. If any appear under a Disallow, that platform cannot cite you. Some hosts and CDNs also block at the firewall level, which robots.txt will not reveal.
Ryan Goering
CEO & Founder, BaaDigi
U.S. military veteran and digital marketing strategist who built BaaDigi to help contractors generate predictable leads and revenue. 15+ years in SEO, PPC, and AI-powered marketing automation.