Google announced it is changing the user agent name of the current GoogleBot in December. We knew this was coming when Google launched the new Evergreen GoogleBot. Google didn't change the user agent ...

Google has announced that its mobile crawler now has a user-agent that’s specific to smartphones. The company has already had a user-agent that targets feature phones. The new Smartphone ...

Google announced that starting in December Google will update Google’s user agent to represent the version of Chrome that GoogleBot is running. It is important to update any code you may have that may ...

There have been an ongoing discussions over the past few weeks across social media that Googlebot has dramatically reduced its crawling. For example, the founder of a web crawl analysis service ...

Google’s John Mueller answered a question on Reddit about Googlebot crawling and the completeness of the snapshot. The person asking the question received a response that touched on edge cases and ...

There is no doubt that a ton of bot activity on one's sites are from rogue spiders. Spider or bots that pretend to be legit bots but are there to steal your content. We have covered several sessions ...

In December 2011 Google announced that it now has a smartphone Googlebot-Mobile. Historically, Googlebot-Mobile has concerned itself only with mobile sites designed for feature phones, so this is a ...