About the AI Governance Rule Watch™ crawler

This page explains the automated reader that visits government and regulatory websites on behalf of AI Governance Rule Watch™. If you found AIRuleWatchBot in your server logs, this is who we are and what we were doing.

What the crawler does

AI Governance Rule Watch™ monitors public regulatory information (statutes, bills, rules, executive orders, court rules, standing orders, agency guidance, and official announcements) so businesses can understand how artificial-intelligence governance affects them. The crawler checks official pages for changes and records when something meaningful changed. It never republishes a page's content; it always links back to the official page as the source of truth.

How it identifies itself

Every request carries this identifier, which links back to this page:

Mozilla/5.0 (compatible; AIRuleWatchBot/1.0; +https://airulewatch.app/bot)

The crawler never disguises itself as an ordinary human browser, and it never attempts to access private, logged-in, or members-only content.

The rules it follows

It reads and obeys each site's robots.txt rules, including any crawl delay, and re-checks them daily.
One request at a time per site, at least two seconds apart.
It asks "has this changed since last time?" on every visit, so unchanged pages cost the site almost nothing.
If a site says it is too busy (429 or 503), it waits exactly as long as the site asks before returning.
Where a site offers an official API or feed, that is used instead of reading the page.
If a site's rules say a page is off-limits to automated readers, the crawler stops and does not work around it.
It never attempts to bypass CAPTCHAs, logins, paywalls, or any other access control.

For website administrators

If you administer a website we monitor and would like us to slow down, skip a page, or stop visiting entirely, the fastest option is your robots.txt file. We honor it automatically. You can also reach the team through the contact options on this website, and we will adjust or retire the source promptly.