|Your everyday productivity workspaceBuild a valid robots.txt with user-agent rules, allow/disallow paths, and sitemap URL. Copy or download instantly.
Last updated: June 2026
User-agent: *
Our free robots.txt Generator builds a valid robots.txt file with user-agent rules, allow/disallow paths, crawl-delay, and sitemap URL — with live preview.
One-click presets for WordPress, Next.js/Vercel, allow-all, block-all, and AI crawler blocking (GPTBot, CCBot).
Copy to clipboard or download as robots.txt instantly. No signup, runs entirely in your browser.
Read the full guide: robots.txt generator guide — syntax & validation
Create a robots.txt before going live.
Prevent crawlers from indexing /admin/ or /wp-admin/.
Add GPTBot, CCBot, and Meta AI blocking rules.
Apply standard _next and /api/ disallow patterns.
Add your sitemap.xml URL for search engines.
Your feedback helps us build better tools.