When a search engine arrives at a site, it behaves like a visitor in a building without a floor plan. It opens doors, follows corridors, and sometimes misses entire rooms. The sitemap is that plan: a file that lists the addresses of your site so that Google, Bing, and others know exactly what to explore.
XML Sitemap and robots.txt: two complementary files, not interchangeable
The robots.txt file tells bots what they should not explore. The sitemap does the opposite: it signals what you want to be discovered. Confusing the two is like mixing a list of prohibitions with an invitation map.
In practical terms, the sitemap does not force a page to be indexed. Google specifies this in its official documentation: this file facilitates the discovery of URLs, but the decision to index remains with the engine. If a page is blocked by robots.txt or returns an error, its presence in the sitemap will change nothing.
For the file to be taken into account, two methods still work: declaring it in Google Search Console or adding its path in robots.txt. The old “ping” mechanism (automatic notification via a dedicated URL) has been abandoned by Google since 2023.
A concrete example of a well-structured sitemap can be found on the sitemap page of L’Igor Web, which groups the site’s URLs in a format readable by engines.
Lastmod tag in a sitemap: what Google really uses
Have you ever noticed the lastmod tag in a sitemap file? It indicates the last modification date of a page. On paper, it helps engines prioritize their exploration by visiting recently updated content first.
In practice, Google only uses this date if it is consistently accurate. If you regenerate your sitemap every night and all dates change to “today” without any real content modification, the engine eventually ignores the information. The date must correspond to a significant change: rewritten text, added structured data, modified internal links.
Updating only the copyright mention in the footer or changing a word in the footer does not constitute a significant modification. CMSs that automatically regenerate the sitemap with updated dates at each build create exactly this problem.
- A reliable lastmod date helps Google explore your priority pages faster
- A falsified or automated date unrelated to the content will eventually be ignored
- The
priorityandchangefreqtags, once popular, are no longer used by Google for its exploration decisions
Sitemap and IndexNow on Bing: two mechanisms not to oppose
Google captures most of the attention, but Bing treats sitemaps differently in one respect. The IndexNow protocol, supported by Bing (and other participating engines), allows real-time signaling that a page has just been created, modified, or deleted.
The sitemap remains a periodic inventory. IndexNow works like an instant notification. Both complement each other instead of replacing: the sitemap provides the overview, IndexNow accelerates the consideration of one-off changes.
Bing Webmaster Tools also offers a coverage report related to the sitemap. This report compares the submitted URLs with those actually explored and indexed. If you submit 200 URLs and only 120 appear as indexed, the report helps distinguish a discovery problem (the bot cannot find the page) from an indexing problem (the bot finds the page but chooses not to index it).
Creating an XML sitemap on WordPress or manually
On WordPress, the CMS generates a sitemap by default since version 5.5. This native sitemap covers pages, posts, categories, and tags. For a simple site, this is sufficient.
Specialized plugins (Yoast SEO, Rank Math, XML Sitemaps) add options: exclude certain types of content, segment the sitemap by category, or manage a sitemap index for large sites.
A sitemap index groups multiple sitemap files under one. Each individual file has a size and URL limit. The sitemap index allows you to exceed this constraint by organizing URLs by section (articles, products, pages).
For a static site or custom development, manual creation is still possible. The file is standard XML with a simple structure:
- A root tag
urlsetthat encompasses everything - A
urlblock for each address, containing at least theloctag with the full URL - Optionally, the
lastmodtag with a date in ISO 8601 format - The file must be encoded in UTF-8 and publicly accessible, usually at the root of the site
Diagnosing a sitemap that doesn’t work in Search Console
Google Search Console displays the status of your sitemap after submission. Two pieces of information matter: the number of detected URLs and the number of indexed URLs. A significant gap between the two signals a problem to investigate.
Common errors are often trivial. URLs with redirection or 404 errors in the sitemap pollute the file and dilute its usefulness. A sitemap should only contain pages returning a 200 code (accessible page) that you actually want to see indexed.
Password-protected pages, internal search result pages, and faceted filter pages on e-commerce: none of this belongs in a sitemap. Each URL present in the file sends a signal to the engine: “this page deserves to be explored.” If you include pages without value, you waste the crawl budget that the bot allocates to your site.
The coverage report in Search Console then becomes a real technical diagnostic tool, well beyond a simple acknowledgment. Regularly comparing submitted and indexed URLs reveals flaws in your architecture before they affect your visibility in search results.
