GoogleBot Update Crawls Modern Web Features

Google has announced the latest update to their GoogleBot crawler and reports it has been designed to crawl modern websites with less problems. This includes being able to crawl JavaScript features and lazy loading of images. Because of the potential impact on blocking content from being indexed, these have long been a concern in SEO and widely considered to be poor for SEO. This update shows Google's commitment to making sure the modern web is fully crawlable and indexedable and allows developers and webmasters to use techniques that offer benefits to improve page load times and user experience. GoogleBot is used by Google to crawl pages and include them in Google's index. This new update makes GoogleBot evergreen with the latest version of Chromium, the open-source browser that Google Chrome is built on. In other words, GoogleBot will now always run the latest Chromium rendering engine when rendering pages for search, and will regularly update its rendering engine to ensure support for latest web platform features. According to Google, this update means that GoogleBot now supports over a thousand new features. Examples given included ES6 and newer JavaScript features, IntersectionObserver for lazy-loading, and Web Components v1 APIs. Accordingly, GoogleBot can now crawl more modern websites with less problems. Google was quick to clarify, however, that GoogleBot is still not perfect. For instance, it still needs to do two waves of crawling to render JavaScript, so there will be a delay in indexing some sites because Google can't crawl and index all of the content in one pass. Therefore, despite GoogleBot's improved capabilities, it is still worth pursuing on-site SEO aimed at making content more easily indexable. The rollout of the new GoogleBot was not instantaneous. It actually began last November, at which time only 1% of the web was affected, and slowly continued until finally reaching 100% this month. In response to questions about whether the rollout of the new GoogleBot was related to the recent de-indexing bug, Google said that the two were unconnected. References "The new evergreen Googlebot" by Martin Splitt. Google Webmaster Central Blog. May 7, 2019. https://webmasters.googleblog.com/2019/05/the-new-evergreen-googlebot.html "Google’s web crawler Googlebot now runs on the latest version of Chromium: Why that’s a good thing" by Barry Schwartz. Search Engine Land. May 7, 2019. https://searchengineland.com/google-will-ensure-googlebot-runs-the-latest-version-of-chromium-316534 "GoogleBot Runs Latest Chrome - This Is Big News" by Barry Schwartz. Search Engine Roundtable. May 8, 2019. https://www.seroundtable.com/googlebot-latest-chromium-27535.html "Google: New GoogleBot & Indexing Bugs Unrelated" by Barry Schwartz. Search Engine Roundtable. May 14, 2019. https://www.seroundtable.com/new-googlebot-unrelated-indexing-bugs-27561.html "Google Rolled Out The New GoogleBot Over Several Months" by Barry Schwartz. Search Engine Roundtable. May 14, 2019. https://www.seroundtable.com/new-googlebot-rollout-happened-over-months-27560.html