Select Page

Over the last few months we noticed an uptick in website owners and some content delivery
networks (CDNs) attempting to use 404 and other 4xx client errors (but not 429) to attempt to
reduce Googlebot’s crawl rate.

​Google Search Central Blog  

Don’t use 403s or 404s for rate limiting Google Search Central Blog