Robots.txt, Sitemap and .htaccess Generator
Generate three common technical SEO files with readable settings, validation and one-click downloads.
Only include canonical URLs you want indexed. A single sitemap supports up to 50,000 URLs and 50 MB uncompressed.
These rules target Apache 2.4 per-directory `.htaccess`. Test in staging; server configuration can differ.
Generate Technical SEO Files Carefully
A robots.txt file controls crawling, not guaranteed de-indexing. XML sitemaps should use absolute canonical URLs. Apache rewrite behavior differs between virtual-host and `.htaccess` contexts, so review and test generated rules before production deployment.
Frequently Asked Questions
Can robots.txt remove a page from Google?
Not reliably. Use appropriate access controls or a noindex directive where supported, and do not block crawling when a crawler needs to see the noindex.
Should sitemap URLs be absolute?
Yes. Use fully qualified canonical URLs including the protocol and hostname.
Will every .htaccess file work on every server?
No. `.htaccess` is for Apache when overrides are enabled. Nginx and other servers use different configuration syntax.