PageChangeDetectorBot
How the Page Change Detector checks behave when they visit a site, and how to stop them.
The user agent string
Every request identifies itself with this exact string:
PageChangeDetectorBot/1.0 (+https://pagechangedetector.com/bot)There is no second user agent and no browser one. A site that wants to allow or refuse us can match this string in robots.txt or at its server.
How often we fetch
A monitor is checked at the interval its owner chose, and the server refuses an interval faster than the plan allows. The fastest interval is 60 minutes on Free, 15 minutes on Pro and 10 minutes on Business. A browser-rendered monitor is never checked faster than every 30 minutes on any plan. One check asks for one page: the page the monitor watches.
At most one request per host every 5 seconds
The gap is shared by every account on the service, so a host with monitors from several customers is not asked more often because of it. We make at most one request to a host every 5 seconds, and at most 2 requests to it at once. Asking for the same URL twice within 60 seconds is answered from the copy we already took, rather than asking the site a second time.
robots.txt
Groups addressed to PageChangeDetectorBot in robots.txt are obeyed, and one is cached for a day. This is not a site crawler: a check fetches one page, the page a monitor watches, so a generic group written for all user agents is not applied to us. A group that names PageChangeDetectorBot is the opt out that always works.
User-agent: PageChangeDetectorBot
Disallow: /How to opt out
Add the robots.txt group above, which refuses every path we would ask for. Or write to [email protected] and we will add your host to our block list. Requests from site owners, hosting providers and site operators are all acted on.
Bot protection
We never bypass bot protection. We do not send cookies, we do not log in, we do not solve CAPTCHAs, and we do not imitate a browser to get past a wall. When a page answers with a challenge or a refusal, the check is recorded as blocked, never as a change, and we do not work around it.
What a request can reach
Only http and https, only ports 80 and 443, and only public addresses. Private, loopback, link-local and internal addresses are refused on the first hop and again on every redirect, so a page cannot use a redirect to point us inward.