List of Googlebot IP addresses released

Posted On 12 Nov 2021
Comment: Off

Knowing which bot is really Google and which is not can help you decide which rogue bots to block from your site.

Why do I need this. If you believe someone is crawling your site, pretending to be Googlebot, and you want to verify that blocking this fake Googlebot crawler is not really Google, then you can use either the command line method or automated IP list method.

IP list. You can access the full list of Googlebot IP addresses over here in this JSON file, note that Google may update this list, so you should check the file probably daily. You can also access all other Google crawlers that match the crawler’s IP address against the complete list of Google IP addresses. Again, these IP addresses may change over time, so you should build into your mechanisms a way to update the IP addresses on your end.

Why we care. Often sites can be slowed down and potentially even go offline because of fake bots crawling and spidering the site. You rarely want to block Google from crawling your site because that can lead to indexing and ranking issues in Google Search. So knowing which bot is really Google and which is not, can help you decide which rogue bots to block from your site.

About the Author