An XML sitemap is a structured XML file that lists the URLs on your site you want search engines to discover, along with hints about when each one last changed. It doesn't force indexing, but a clean sitemap removes ambiguity and speeds up discovery β especially for large sites, new sites with few inbound links, or sites that publish content frequently.
Direct answer: A valid XML sitemap contains only canonical, indexable, 200-OK URLs, wrapped in the correct urlset (or sitemapindex) namespace, encoded in UTF-8, and kept under the 50,000-URL / 50MB per-file protocol limits. Common validation failures are malformed XML syntax, URLs that don't match the sitemap's declared domain, orphaned pages that exist in the sitemap but aren't linked anywhere on the site (or vice versa), and stale <lastmod> values that no longer reflect real content changes. Submitting a sitemap does not guarantee indexing β it only helps crawlers find and prioritize the URLs you've already decided are worth ranking.
How XML Sitemaps Work
When Googlebot or another crawler fetches your sitemap, it doesn't treat every URL as an instruction β it treats the list as a set of candidate URLs, then applies its own judgment about crawl priority, quality, and uniqueness before deciding whether to index each one. That's why two sites with identical sitemap structures can have very different indexing rates: the sitemap is a signal, not a guarantee.
Sitemaps are typically referenced from robots.txt with a Sitemap: directive and also submitted directly through Search Console or Bing Webmaster Tools. Both paths matter β the robots.txt reference helps any crawler discover the file organically, while manual submission gives you reporting on how many submitted URLs were actually indexed.
Standard XML Sitemap Structure
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>https://example.com/blog/core-web-vitals-guide</loc>
<lastmod>2026-08-17</lastmod>
</url>
<url>
<loc>https://example.com/robots-txt-generator</loc>
<lastmod>2026-09-01</lastmod>
</url>
</urlset>
Every <url> block requires a <loc> element containing the absolute, canonical URL. <lastmod>, <changefreq>, and <priority> are optional; only <lastmod> carries meaningful weight with major crawlers, and only when it's accurate. Sitemaps that stamp every URL with today's date regardless of whether the content changed train crawlers to distrust the field entirely.
Protocol limits
| Constraint | Limit |
|---|---|
| URLs per sitemap file | 50,000 |
| Uncompressed file size | 50 MB |
| Encoding | UTF-8 |
| Nesting | A sitemapindex file can reference multiple urlset files |
If a site exceeds either limit, split the URLs across multiple files and reference them from a single sitemap index:
<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<sitemap><loc>https://example.com/sitemap-pages.xml</loc></sitemap>
<sitemap><loc>https://example.com/sitemap-blog.xml</loc></sitemap>
</sitemapindex>
Common Validation Errors and Fixes
| Error | Cause | Fix |
|---|---|---|
| Malformed XML | Unescaped & in URLs, missing closing tags |
Encode & as &, validate against the sitemap XSD |
| URL not on this host | Sitemap for example.com lists www.example.com URLs |
Sitemap URLs must share the same host as the sitemap file |
| 404 / redirect in sitemap | Deleted or moved pages never removed from the feed | Regenerate the sitemap on publish/unpublish, not on a stale schedule |
| Noindex URL included | A page carries noindex but is still listed |
Exclude any URL with noindex or a canonical pointing elsewhere |
| Orphan pages | Pages exist and are indexable but have zero internal links pointing to them | Add contextual internal links; sitemaps shouldn't be the only discovery path |
Finding and Fixing Orphan Pages
An orphan page is technically live and crawlable, but no other page on the site links to it β so users and crawlers can only reach it through the sitemap or a direct URL. That's a weak position: internal links pass context and authority, and a page with none of either signals low importance even if it's listed in the sitemap. To find orphans, cross-reference your sitemap URL list against a full site crawl's internal-link graph β any URL that appears in the sitemap but never appears as a link target on another page is an orphan. The fix is nearly always adding relevant contextual links from related content, not removing the page from the sitemap.
Sitemap Hygiene Checklist
- List only canonical, 200-OK, indexable URLs. Exclude redirects, 404s, and
noindexpages. - Strip tracking parameters. URLs like
?utm_source=newslettercreate duplicate entries for the same canonical page. - Keep
<lastmod>honest. Update it only when the content meaningfully changes, not on every deploy. - Regenerate automatically on publish. A sitemap built once and never updated drifts out of sync with the live site.
- Reference it from
robots.txt. AddSitemap: https://example.com/sitemap.xmlso any crawler can find it without manual submission.
You can validate structure and check for broken or non-canonical entries with our XML sitemap validator, confirm the Sitemap: directive is correctly declared with the robots.txt tester, and spot-check individual URL status codes with the HTTP status checker.
Frequently Asked Questions
Do <priority> and <changefreq> actually affect rankings?
No. Google has stated these fields are largely ignored, since they're self-reported and easy to game. Focus effort on <loc> accuracy and honest <lastmod> values instead.
Should my sitemap include paginated or filtered URLs? Generally no. Include the canonical version of a content set, not every parameter combination. Faceted or paginated variants usually belong outside the sitemap unless each one is independently valuable and indexable.
How often should I regenerate my sitemap? Ideally automatically, whenever a page is published, updated, or unpublished β not on a fixed weekly or monthly cron job that can lag behind real site changes.
Can a sitemap fix a crawl budget problem?
It helps prioritize, but it can't compensate for a site with excessive low-value or duplicate URLs. Pair sitemap hygiene with robots.txt disallow rules and canonical tags for a complete crawl-budget strategy.
Does every site need an XML sitemap? Small sites with strong internal linking and few pages benefit less, but there's little downside to having one. For sites with more than a few dozen pages, or with weak internal linking, a sitemap meaningfully speeds up discovery.