Hear about a broken or shrinking sitemap before search engines do
SEOComet reads your sitemaps and sitemap indexes every hour on Free, down to every 30 minutes on Business, counts their URLs and lastmod dates, and alerts you when one goes missing, stops parsing, loses URLs or stops updating. Google News sitemaps are read more often: every hour on Free, down to every 10 minutes on Business.
What it catches
Sitemaps fail quietly. A plugin update writes broken XML, a cache serves an HTML page in the sitemap’s place, or a deploy drops half the URLs. These are the issues SEOComet raises.
| Issue | Severity | What it means |
|---|---|---|
| Sitemap missing | Critical | The sitemap was there before, or robots.txt lists it, and now the address answers 404 or 410, so search engines can’t read it either. One that never existed raises nothing. |
| Sitemap can’t be read | Critical | The file isn’t valid XML, or its root is neither urlset nor sitemapindex. A plugin update, or a cache serving an HTML page instead, are the usual causes. |
| Sitemap is empty | Critical | The sitemap was read correctly but lists no URLs at all, which usually means a build or plugin failed. |
| Fewer URLs in the sitemap | WarningCritical Critical past twice your threshold | The sitemap lists far fewer URLs than on the last complete check. The threshold starts at a 30% drop and you can change it. Not raised for news sitemaps. |
| A child sitemap failed | Warning | A sitemap index lists other sitemaps, and at least one of them couldn’t be fetched or read. Their URLs may drop out of search. |
| Couldn’t check the address | Warning | The sitemap answered with an error other than 404 or 410, or the request timed out or failed. |
| Sitemap looks stale | Info | The newest lastmod date is older than your limit (30 days to start with), so the sitemap may have stopped updating. Not raised for news sitemaps. |
| Index listed inside an index | Suggestion | A sitemap index lists another index, which Google skips there. Nothing is lost if the inner index is also in robots.txt or Search Console. Shown on the check, with no incident or alert. |
How it reads a sitemap
SEOComet fetches the sitemap, checks the XML, and counts the URLs and the newest
lastmod. It doesn’t visit the URLs, except the newest articles in a
news sitemap, and it doesn’t store the file, only the counts and dates.
For a sitemap index it fetches the child sitemaps one at a time, up to 200 per check. Child sitemaps on another site aren’t fetched and count as failed. If a child fails, or the check runs out of time or reaches the cap, the count is marked incomplete. Only complete counts are compared, so a slow night doesn’t look like a lost half of your site.
Sitemaps can be up to 50 MB each and get 30 seconds to load.
- Where sitemaps come from
-
The
Sitemap:lines in robots.txt for your domain and its subdomains, up to 50, or/sitemap.xmlif there are none, watched quietly until it exists. Add any other sitemap on your domain or its subdomains by hand. - URL count drop
- A warning past 30%, critical past 60%. Set your own threshold on each sitemap’s Settings tab.
- Stale after
- 30 days without a newer lastmod. Raise it for a site that rarely changes.
- Check interval
- Every hour on Free and Pro and every 30 minutes on Business.
Google News sitemaps
A sitemap with Google News tags is checked as a news sitemap, with no setup. Google News only reads articles from the last 48 hours, so the checks are about freshness, not size.
| Issue | Severity | What it means |
|---|---|---|
| News sitemap stopped updating | Critical | The newest article is older than your limit (6 hours to start with). Google says to add articles to the news sitemap as they're published, so new stories may not be reaching Google News. Google: News sitemap best practices |
| New article not indexable | Warning | One of the 5 newest articles didn’t load, is set to noindex, or has a canonical that points to another address, on two checks in a row. Google: Block indexing with noindex , Canonical URLs |
| Articles older than 48 hours | Warning | Google says a news sitemap should only list articles from the last two days. Older ones should come out, or lose their news tags. Google: News sitemap best practices |
| Too many URLs for a news sitemap | Warning | Google allows at most 1,000 news entries in one news sitemap and says to split a bigger one into several. Google: News sitemap reference |
| News tags missing | Warning | Some URLs lack a tag Google requires: the publication’s name or language, the title, or a valid publication date. Google: News sitemap reference |
- Stale after
- 6 hours without a new article to start with. Set 1 to 72 on the check’s Settings tab.
- Newest articles
- Each check opens the 5 newest articles to make sure they load, aren’t noindex and don’t point their canonical somewhere else.
- Check interval
- Every hour on Free, every 15 minutes on Pro and every 10 minutes on Business.
A news sitemap shrinks every weekend as old articles leave it, so the URL count drop and lastmod checks don’t apply to it.
Join the SEOComet waitlist
SEOComet isn’t open to the public yet. Leave your email and we’ll tell you when you can add your sites. The SEO Toolbar Chrome extension is free and works today.