Sitemap checks

A sitemap check reads one sitemap, and every sitemap it lists if it’s an index, and counts the URLs. It tells you when the sitemap goes missing, can’t be read, is empty, shrinks suddenly or stops being updated.

SEOComet adds a sitemap check for each Sitemap: line on your own domain (up to 50, included on every plan), or for /sitemap.xml when robots.txt lists none. If /sitemap.xml isn’t there, its check shows Not found and sends no alerts until it appears (see Missing files). You can add others from Add check on the site’s Checks page. A sitemap must be on the site’s own domain or one of its subdomains, so for example.com both example.com/sitemap.xml and blog.example.com/sitemap.xml are fine.

How often

Every hour on Free and Pro and every 30 minutes on Business. A sitemap with Google News tags is read more often: see Google News sitemaps.

Sitemap indexes

A sitemap is either a list of pages (a urlset) or a list of other sitemaps (a sitemapindex). When the check reads an index, it also fetches each sitemap the index lists, one after another, and adds up their URLs. It reads up to 200 of them per check.

• A listed sitemap that can’t be fetched or read raises A child sitemap failed, naming the ones that failed.
• Listed sitemaps on another domain aren’t fetched, and count as failed.
• An index listed inside another index is a suggestion. Google skips it there, so SEOComet doesn’t count its URLs either. Nothing is lost if that inner index is also in robots.txt or submitted in Search Console.

When a listed sitemap fails, or some aren’t read because of the limit or time, the URL count is incomplete. SEOComet then doesn’t judge the count: no Sitemap is empty, no URL drop and no stale check on that run.

Fewer URLs

Each check compares its URL count with the previous check’s. If the count dropped by more than your threshold, that’s a warning. If it dropped by more than twice the threshold, it’s critical. The threshold starts at 30%, so a drop of more than 60% is critical.

• The comparison only happens when both counts are complete. A check that couldn’t read every listed sitemap isn’t compared, and the next check isn’t compared with it.
• Because each check compares with the one before, the incident closes on the next check that gets an answer, even if the URLs don’t come back. The new, lower count is then the one compared with.
• News sitemaps don’t get this issue. They only list recent articles, so their count goes up and down on its own.

Change the threshold in the check’s Settings as Warn when the URL count drops by more than (%).

Stale sitemaps

A sitemap looks stale when its newest lastmod date is older than your limit, which starts at 30 days. For an index, that’s the newest date across every sitemap it lists. Dates more than a day in the future are ignored.

This is info, not a warning: a site that rarely changes can have an old lastmod and be fine. If that’s your site, raise the limit in the check’s Settings as Flag as stale when the newest lastmod is older than (days). A sitemap with no lastmod dates is never stale. News sitemaps have their own freshness rule instead.

Size and time limits

• Each file can be up to 50 MB, and each fetch can take up to 30 seconds. A bigger or slower file raises Couldn’t check the address, or A child sitemap failed for a sitemap listed in an index.
• A file that times out, can’t be connected to or answers with a server error or 429 is tried again once 2 seconds later, when there’s time left, before it counts as failed.
• Gzipped sitemaps, such as sitemap.xml.gz, are read too. The limit applies to the unzipped size.
• An index check has 4 minutes in total for all its sitemaps. Sitemaps it doesn’t get to are skipped on that run, and the count is marked incomplete.
• The check stores the URL count and the newest date, not the file, so sitemaps have no Versions tab.

What it can find