Robots.txt Generator

Generate a robots.txt file to control search engine crawlers.

Allow all crawlers by default

Disallow paths (one per line)

Sitemap URL

robots.txt

User-agent: *
Allow: /
Disallow: /admin
Disallow: /private
Sitemap: https://example.com/sitemap.xml

More free tools from Website Management Tools

Related tools