Last updated: September 30, 2026

What this is

Siteavail makes standby copies of websites. To do that well, we need to know how often ordinary business sites actually go down and what those outages look like. The Observatory checks a panel of websites around the clock and records what it finds.

If you're here because you saw us in your server logs, this is the user agent our checks send:

SiteavailObservatory/2.0 (+https://www.siteavail.com/observatory)

How the checks work

  • One page, every ten minutes. We request only the homepage of each site, usually once every 10 minutes. We read at most the first 16 KB of the response, about the size of a small image, and then close the connection. We don't follow links or crawl the rest of the site.
  • Failures are confirmed before they count. If a check fails, we retry about once a minute. We also check from a second network of our own and from a few Globalping probes on home internet connections. Those requests come from Globalping's probes, not from our servers. A site is only marked down when the failure shows up from more than one place, so a problem on our end doesn't get recorded as an outage of yours.
  • One screenshot per outage. If a site stays down for 15 minutes, we load its homepage once in a headless Chromium browser and take a screenshot of what visitors are seeing. That single page load fetches images, stylesheets, and scripts the way a normal browser would.
  • One more look after recovery. When a significant outage ends, we take a second screenshot of the recovered homepage. We also read the homepage, up to three contact or about pages it links to, and /.well-known/security.txt to work out who runs the site.
  • We back off when asked. If a site answers with a 403 or 429, or puts up a bot challenge, we check it less often and eventually stop checking it at all.

What we keep

For a failed check: the HTTP status or error, the response time, and a short excerpt of the response. The excerpt is enough to tell a server error from a suspended-account page or an expired certificate. For a check that succeeds, we keep only the timestamp. We also keep the outage screenshots, and after a significant outage, the recovery screenshot and whatever contact details we found. When a site joins the panel, we note a few things from its homepage, like its title and the platform it runs on.

We don't publish any of this. Outage screenshots go to the owner of the site, if we write to them, and nowhere else.

Why we run it

Siteavail exists because websites go down. The Observatory tells us how often that really happens and what it looks like when it does.

Sometimes, after a site has had a significant outage, we email the owner to tell them what we saw and how a standby copy would have kept their site up. We only send that email after the site is back, and every one has a working unsubscribe link.

Opting out

Email [email protected] with the domain name and we'll stop checking it within 48 hours. Opt-outs are permanent. We keep a suppression list, and it survives when the panel is rebuilt.

Questions

The Observatory is run by Siteavail (MRN24 LLC). If you have a question about the method, want to see the data, or anything else, write to [email protected].