SEO Newsletter Issue 59 – 2022.06.16–2022.06.30

[Official] Google Documentation Update—Googlebot Crawls Only the First 15 MB

Google recently updated several documents. One new detail was that Googlebot crawls and indexes only the first 15 MB of an HTML file, ignoring anything beyond that point. The addition sparked a lot of discussion, but the headline may have made it sound more dramatic than it was. In practice, an HTML file larger than 15 MB is extremely rare.

According to HTTP Archive data, among eight million websites, only 151 homepages were larger than 15 MB uncompressed, while the median webpage was only about...30 KB. Besides, if the HTML alone exceeds 15 MB, SEO is probably not the only issue—the site's entire design framework likely deserves a thorough review.

The update drew so much attention—Gary even published an article in response—mainly because many people mistakenly thought the 15 MB included all webpage subresources, such as CSS, JavaScript, and images. In fact, the limit applied only to the HTML text itself.

Links: Googlebot 15 MB coverage by Search Engine Land / Example of a webpage larger than 15 MB

[Info] Free Recordings from the 2022 SMX Advanced Conference

This year's SMX Advanced took place in June, and all its recordings are now available to watch free on the website after registering for an account. Below are a few talks that interested me and the reasons why. There are also many videos about paid search and digital marketing—around 50 in total—so I will leave those for interested readers to explore.

• What you can learn from the largest link rot study ever performed
  The speaker is Patrix Stox, an experienced SEO professional I regularly follow.
  
• 91 page one rankings in 6 weeks: How pillar-based marketing is changing SEO
  I wanted to hear more about this method and learn what “pillar-based marketing” means.
  
• The May 24th Google Marketing Livestream: Reactions from the experts
  This one is more advertising-related. I had only read written coverage of this year's Google Marketing Live and did not remember anything particularly striking, so I wanted to hear what the experts had to say.
  
• Test web performance tweaks to boost Core Web Vitals rankings
  I am interested in any topic involving web performance and hoped to pick up a new technique.
  
• The declaration of SEO: 6 fundamental truths to live by
  The speaker is Bruce Clay, a very experienced SEO professional. The topic itself did not especially appeal to me, but I had long known his reputation and could not recall ever seeing him speak.
  
• Going beyond 15%: The importance of accessibility in 2022
  I wanted to strengthen my knowledge of web accessibility.
  
• Google, cybersecurity, and you: Being security savvy as an SEO
  Same idea—I wanted to build up my cybersecurity knowledge.

Link: SMX Advanced event page

[Knowledge] How Algorithms Rank Webpages, by Search Engine Neeva

Neeva is an ad-free, privacy-focused search engine whose business model relies on paid memberships offering advanced features. One of its founders is a former Google Ads executive. The company recently shared some information about search engine rankings on Twitter. I found it quite interesting as an accessible introduction to a few concepts behind ranking algorithms.

First, the best-known method for ranking documents by their relevance to a query is TF-IDF. Put simply, the more often a word appears in one document, the more important it is. But when that word appears in every document—for example, words like “I,” “you,” “he,” and “the”—the less important it becomes.

This creates a problem. A very long document may cover many different topics, leaving it with a low TF-IDF score for every word, but that does not mean its content on a particular subject is poor. A document length normalization step is therefore usually applied so that long documents are treated fairly too.

Even that does not solve every problem. For example, when someone searches for neeva, should neeva.com or neevaneevaneeva.com rank higher? Even after processing, their relevance is the same. Information beyond the text is needed to make that decision. An important principle of ranking algorithms is that how others see you matters more than how you see yourself.

Information beyond the text includes the backlinks everyone knows about, but query-click data is also very important. Neeva records the query and webpage URL associated with every click on its search engine and uses that data as one of its ranking signals.

Links: Introduction to Neeva / Neeva's website / Twitter thread

[Report] The 2022 State of Link Building Survey

I also covered this annual survey in the newsletter around the same time last year. This year's report collected the views of 270 SEO professionals. I had not worked with link building for a while, so I found the report quite interesting. It was not too long and was easy to digest. Below are a few findings that stood out to me.

• The five most commonly used link-building tools were Ahrefs, Google Sheets, SEMrush, HARO, and Screaming Frog.
• The three most common metrics for measuring link building were rankings, search visibility, and the number of linking domains.
• Roughly how long did it take, on average, to earn one link? 32% answered one to two hours, while 24% answered three to five hours.
• 69% of respondents included nofollow links in their reports.
• 69% of respondents believed buying links had a positive effect on rankings.
• 86% of respondents said their primary reason for producing content was to earn links.
• The two content formats considered most effective for earning links were long-form, report-style content and interactive content.

Link: The State of Link Building Report 2022

[Practical] Run Screaming Frog in the Cloud and Control It Remotely

I am pretty surprised to see Screaming Frog publish its own tutorial on running the crawler in Google Cloud. Running it in the cloud means multiple people can use it with a single license key. Another advantage is that crawling a large website will not consume so many resources on your own computer.

Running it in Google Cloud is not free. The cost depends on how much you use it and the performance of your virtual machine, but crawling 3.1 million URLs over two days costs only about €20. New users also receive US$300 in credit.

One of my coworkers tested the process firsthand. Although it sounded complicated—setting up Google Cloud, creating a virtual machine, remotely controlling that virtual machine, and so on—he said he had it working within about one to two hours. I also helped him test the remote-control part, and we got that working within about 15 minutes.

This approach can save quite a bit of money by letting multiple people share an account, but it also has drawbacks. Remotely operating a cloud machine is often laggy, and moving the results from the VM back to a local machine adds a few steps. Still, if you have a use for it, I recommend giving it a try!

Link: Original Screaming Frog article

[Recommended Reading] The SEO Career Skills Tree

I very, very strongly recommend reading this SEO skills tree put together by Tom Critchlow. It separates agency-side and in-house skills into five levels, divides them between soft and hard skills, pairs them with job titles and average U.S. salaries, and offers examples to help you determine where you fit.

After spending some time in SEO, many people feel uncertain about where to go next. I have found myself in that dead end more than once. Sometimes it may be a good thing, because it means you are ready to move into the next stage. I recommend looking at the advice in the skills tree. Below are two other articles by Tom that I covered before.

Links: SEO reporting for executives / Advocating for and scaling SEO in-house

[Technical] Chrome v103 Officially Supports Early Hints (Status Code 103)

When I covered Early Hints (status code 103) in Issue 41, this new protocol led by Cloudflare was undergoing implementation testing. A few months later, Cloudflare published an article explaining that the technology can improve website performance by as much as about 30%.

When a browser sends a request to a server, the server typically needs some time—server think time—to prepare before sending a response back. With Early Hints, the server effectively says as soon as it receives the request, “I need some time to prepare the webpage you requested, but you will need this CSS file and this JavaScript file later, so go fetch them first!”

Google recently announced that Chrome would support the technology beginning with version 103. Shopify also helped test it and saw good results. If it interests you, send it to your engineers for discussion. If you use Cloudflare's CDN, even the free plan lets you enable the feature with one click.

[Recommended Reading] Information Gain—the Future Direction of Content SEO

I am usually less interested in articles about content SEO, but I saw quite a few SEO veterans I respect sharing this one on Twitter. Sure enough, its first two paragraphs pulled me right in.

You know the feeling: every topic, every keyword, and every conceivable piece of traffic-driving content has been cornered by a handful of famous brands, whose content is both long and comprehensive. Their content is impeccably optimized for SEO too. And if you are honest with yourself, you will realize their content is not very different from what you wanted to write...

The article then introduces a patent Google received in 2020 that aimed to address the problem of most documents offering similar information. It called the concept “Information Gain”—in other words, rewarding documents that contribute new information to a topic.

The author also offers three methods he thinks are worth trying to increase an article's information gain. I will leave you in suspense and half-force you to read the article yourself. It is genuinely good!

Link: Information Gain

[Video] How Google Rolls Out Algorithm Updates

The question in the video is, “How does Google roll out algorithm updates? Does it always begin with English-speaking countries and then gradually release them in other countries?”

John answers that Google usually wants an algorithm to work across all countries and languages, rather than changing it for individual markets, because that makes it easier to manage and test. In other words, when an update launches, Google tests the most relevant language where necessary. That language might be English, but not always, and the update then gradually expands into other language markets.

[Summary] Podcast—User Experience (UX) and SEO

In this episode, Martin, Gary, and Lizzi discuss user experience. Lizzi is a lead editor and writer for Google Search. She says that before recording the episode, she looked back and found that the official documentation barely mentioned UX, only Page Experience (PX). She thought the term may have been created from a corporate marketing perspective and agreed that PX did not cover the entire scope of UX.

Google's search best-practice documents did include principles such as “write for your readers.” Gary jumps in to say that when a concept is difficult to define, Google's documentation tends to use vague language. For example, “write for your readers” might mean “write for your readers' reading level,” but Google would not use the latter phrasing because reading level is difficult to define precisely. Within UX, however, how easy an article is to read still deserves consideration.

Other examples include “write high-quality descriptive headings” and “site architecture”; Google could not provide a single standard.

Partway through the episode, they discuss content hidden inside tabs or collapsible modules. Martin asks whether, as Google had said before, content not immediately visible to users was less important than directly visible content. Gary answers that Google did use such a mechanism in the past, but no longer did. His answer surprises Martin, and even Search Engine Roundtable published an article specifically to reiterate the point.

Link: Transcript