GPTBot is the web crawler used by OpenAI to gather training data for GPT models. Allowing GPTBot in your robots.txt is how you signal that you want your business cited in ChatGPT answers. Most small businesses unintentionally block AI crawlers; allowing them is a basic AEO step.
The longer version
GPTBot, along with ClaudeBot (Anthropic), PerplexityBot (Perplexity), and Google-Extended (Google AI), are the bots that read your website to inform AI assistants. By default many websites block all crawlers except Googlebot, which means AI assistants never see your content. Adding 'User-agent: GPTBot Allow: /' to robots.txt is a 30-second AEO win that most agencies skip.
Common questions
Will allowing GPTBot affect Googlebot?
No - they are separate crawlers. Allowing one does not affect others.
Should I allow ALL AI crawlers?
For most SMBs, yes. The visibility upside outweighs the marginal cost of crawl bandwidth.
Does Google have its own AI crawler?
Yes - 'Google-Extended.' It is separate from Googlebot. Allow it for Google AI Overviews to cite you.
Need this implemented in YOUR business?
Take the 5-min audit. I'll tell you honestly whether this fits.
Get my free AI audit →