Updates ·

Inspect JavaScript-based Structured Data

The Inspector service can now use a browser to fully render HTML before extracting its structured data. For web pages that rely on JavaScript to dynamically load or generate content, the new Rendered Web Page option provides a more accurate analysis about structured data that eventually arrives to an end user. The final snapshot also includes several other insights that can assist with debugging, such as:

  • Screenshot of the rendered page can offer a quick visual check whether stylesheets and JavaScript may be working as expected.
  • Status Report with details about any unsuccessful responses, bad connections, or blocked requests due to policies such as robots.txt.
  • Web Archive (WARC) containing all the HTTP request and response headers sent during the rendering session, along with additional technical metadata.

Due to the additional processing requirements, a Rendered Web Page snapshot consumes more credits than a basic file retrieval or code upload.

Web Crawlers

Since the Inspector service is focused on data and debugging, it renders with fairly generous time and execution constraints. Support for JavaScript by web crawlers range from no-support to works-with-caveats. Before starting on a new project, be sure to review the latest, official documentation about capabilities, such as Bing's Webmaster Guidelines and Google's JavaScript SEO Basics.