Sitemap

A sitemap is a table of contents for machines. It is a single file on your server, usually at the address sitemap.xml, in which every page of your website is listed with its full address. People do not normally read it; it is meant for the crawlers of the search engines, which look there to see which pages exist at all.

Its use is limited, and precisely for that reason easy to explain: a sitemap makes sure a page is found, not that it is placed well. It replaces neither linking nor content. Anyone selling it as a ranking lever is selling the wrong thing — it is a precondition, not an advantage.

It becomes important in three cases. On large websites, where hundreds of pages cannot all be reached from the homepage within a few clicks. On new websites that hardly anyone outside links to yet. And for pages that are important but poorly linked — detail pages, for instance, that would otherwise only be reachable through a search function.

Besides the address, each entry can carry a date of last change. That date is the only extra Google says it pays any attention to at all — and only when it is accurate. Systems that set every date to "today" on each nightly run devalue the field for the entire website. Details about importance or change frequency, which older guides still recommend, are ignored.

There are fixed limits written down in the protocol: at most 50,000 addresses and at most 50 megabytes per file in its uncompressed state. Anyone with more pages splits them across several files and puts an overview file in front, a so-called sitemap index. For most businesses this is irrelevant — a tradesman's website rarely has more than a hundred pages.

The file is registered in two places. In robots.txt, the small text file in the root directory of your website, one line gives the full address of the sitemap. And in the Google Search Console it can be submitted once; after that it is fetched regularly on its own. Resubmitting after every change is not necessary.

The most common mistakes keep appearing in the same form. The sitemap contains addresses that no longer exist and that end in a 404 error. Or it contains addresses that are at the same time excluded from inclusion by noindex — in which case the website contradicts itself. Or the file points to the www variant while the website is served under the variant without www.

That is why the sitemap is not a filing place for us but a testing instrument: we fetch every address named in it individually and look at what actually comes back. A sitemap that somebody generated once and never looked at again is, as a rule, wrong after the first rebuild. How a website is moved across without loss is described in the guide to relaunching.