Robots.txt Generator
Generate robots.txt with per-crawler rules, sitemap lines and common presets.
Turn a list of URLs into a valid sitemap.xml file.
Add a lastmod date after a comma or tab to override the shared value.
Your list exceeded the per-file limit, so it was split. Upload each numbered file plus this index.
This builds a sitemap from the URL list you supply. It cannot crawl your site, because a page in your browser is not allowed to request another domain.
This generator takes a list of URLs you already have and produces a valid sitemap.xml. It does not crawl your site, because a page running in your browser cannot request another domain's pages; browsers block that. What it does is the other half of the job, and the half that is tedious by hand: correct XML structure, correct date format, and proper escaping of the ampersands and other characters that otherwise make the file invalid.
You can set lastmod, changefreq and priority for the whole list, or supply per-URL values in extra columns. Worth knowing: search engines have said plainly that they ignore changefreq and priority. They are included because the format allows them and some other tools expect them, not because they will change anything.
If your list exceeds 50,000 URLs or the file would exceed 50 MB, the tool splits it into numbered sitemaps and produces the sitemap index file that references them, which is what the specification requires.
The xml sitemap generator is used by writers, developers, students, marketers and anyone else who needs the job done once without installing software. Common cases include:
Because browsers block pages from requesting other domains, which is a security rule, not a limitation we chose. Any online tool that crawls for you is doing it from its own server, which means sending it your site to fetch. This tool works from a list you provide.
From your CMS export, a crawler such as Screaming Frog, your server logs, or an existing sitemap. For a static site, listing the HTML files in your build output works.
Search engines have stated that they ignore both. They remain part of the sitemap format and some tools expect them, so they are supported here, but setting priority to 1.0 on every page achieves nothing.
50,000 URLs or 50 MB uncompressed per file. Beyond that you need several sitemaps and an index file listing them, which this tool generates automatically.
W3C datetime, which in practice means YYYY-MM-DD or a full timestamp with a timezone offset. The tool normalises what you supply. An inaccurate lastmod is worse than none, since crawlers that trust it will recheck pages that have not changed.
At your domain root as /sitemap.xml, then add a Sitemap line to robots.txt and submit it in Search Console.
If the xml sitemap generator is not quite what you need, these other free tools solve closely related problems.
Generate robots.txt with per-crawler rules, sitemap lines and common presets.
Remove repeated URLs, treating trailing slashes and tracking parameters as the same.
Extract all URLs from text or HTML, with anchor text and filtering options.
Generate canonical link tags, with hreflang alternates and common-mistake checks.
Pretty-print XML with proper indentation, and catch malformed documents.