Free tool · runs in your browser
AI robots.txt generator
Choose which AI assistants can read your site. Allow the bots that put you in AI answers, and decide separately about AI training. Get a ready-to-use robots.txt.
Your robots.txt
Upload to your website's root so it opens at yoursite.com/robots.txt.
User-agent: * Allow: / # --- AI assistants --- User-agent: OAI-SearchBot User-agent: ChatGPT-User User-agent: PerplexityBot User-agent: Claude-SearchBot Allow: / User-agent: GPTBot User-agent: ClaudeBot User-agent: Google-Extended User-agent: Applebot-Extended Disallow: /
Keep your robots.txt right, automatically
Now see if AI actually recommends you
The free Eudoxia Score check asks AI assistants the questions your customers ask, and shows who they recommend instead of you.
Questions
Should I block AI training bots?
That's your choice. Blocking training bots (GPTBot, ClaudeBot, Google-Extended, Applebot-Extended) stops your content being used to train models, and doesn't stop you appearing in AI answers, as long as you allow the answer bots.
Will this replace my existing robots.txt?
Paste your current rules into the “existing rules” box and they'll be kept. The AI section is added alongside them.
Where does robots.txt go?
In the root of your website, so it opens at yoursite.com/robots.txt. Many website builders and SEO plugins have a robots.txt editor.
Guides for this tool
More free tools
- AI readiness checker
Score how ready your website is to be read and recommended by AI assistants.
- AI crawler checker
See if your robots.txt blocks ChatGPT, Gemini, Perplexity, Claude or Apple AI.
- Local business schema generator
Create LocalBusiness structured data (JSON-LD) for your website in a minute.
- llms.txt generator
Create an llms.txt file that summarises your business for AI tools.
- FAQ schema generator
Turn your questions and answers into FAQPage structured data and HTML.
- Structured data checker
See which schema.org data a page has, and what's missing or broken.