Last updated July 14th 2024
(New AI component added)

The biggest leak in Google's history just occurred, exposing 14,014 secret internal API references used within its search engine. 

While many people underestimate the impact of this latest Google leak, I believe it allows us to confirm or debunk theories about Google's inner workings, revealing new insights that can revolutionize how we approach SEO.

It allows us to dodge many of the spam traps that Google has set in place and shows us the exact path forward to rank on Google. I've spent hundreds of hours analyzing the leak and cross-referencing it with my 12 years of SEO notes, including insights from ranking, examining search patents, and conducting SEO tests.

I'm excited to share all my discoveries with you.

And I won't stop at just sharing discoveries—I'll also explain how you can implement everything to achieve better SEO results that can significantly impact your bottom line.

1. If you've been affected by a recent Google update or are looking to rank your site in record time, this is for you.

2. To all search engine professionals, SEO agencies and sole-entrepreneurs owners looking to rank your clients' website rankings, this is for you.

The data warehouse leak contains documentation on the APIs that Google uses to build it's algorithms.

These are the building blocks of the search engine.

For example, if one morning a search engineer wakes up and decides he wants to create a new algorithm that only show search snippets from websites with a PR5+ that have more than 20,000 Chrome views and have a Spamscore of 10 or lower.  Then he can retrieve the data using these APIs.

Hypothetical example of how an engineer can use API information create new algorithms. Please note that this is NOT how the search snippet actually works. (I'll explain that later)

This is similar to walking into your favorite restaurant's kitchen while the chef is away. Imagine discovering all his fresh ingredients laying on the counter...

After a few minutes in the kitchen... you can ask:

1. "What is he making here?" 

2. "What kind of toppings is he using when they make pancakes?"

If we know the ingredients, and we know that he's making pancakes... we can easily deduce that the preferred toppings are going to be: blueberries, bananas and strawberries.

This is very similar to ranking factors...

So if we see something like this:

Even though we don't know what the final algorithm is (represented in green), we know that:

1. NoFollow is exclusively related to links

2. We know that Pagerank is ALSO related to links 

3. Therefore, the green block must be covering a link related algorithm.

4. Thus, we can reasonably assume that topicality is also ranking factor when it comes to links. (A measure of how related to the topic the link is.) 

Plus, we have the added benefit that Google employees have often added elaborate descriptions explaining how these APIs fit into the algorithm so sometimes we don't even have to make assumptions, we can just read it!

It should come as no surprise that links still play a major role in rankings, contrary to what Google or some SEOs might claim. They are one of the major ranking factors that help Google understand context, authority and importance.

However... 

With regards to links, I was intrigued to see such a large emphasis on link anchor text.

Here's what I discovered:

Please note that these are partial references to the leak as I do not want to republish the entire document.

According to the documentation, the full description of "context2" confirms that the words before and after your anchor text affect the anchor text.
 While this has long been suspected within the SEO community, it's nice to finally see it in the flesh.

The next element, sourceType, explains how high quality anchor texts come from "Base documents". What this likely means is that  links that come from content ranking in the same keyword bucket, aka highly related content), will carry a heavier weight in terms of anchor text.

Medium quality anchors, according to Google, comes from not very related content. 

Finally, low quality anchors come from, *drum roll*, low quality content.

 While Google doesn't tell us what they consider a low quality document, we can assume that it is content that isn't related to the topic and likely has a host of other metrics that classify it as low quality. (More on this later)

However, the big takeaway is that not all anchors are worth the same!

Ultimately, The BEST links you can get are going to be from OTHER pages currently ranking for the same term.

(Because pages that rank for the same term will be classified in the same bucket)

One big "ah ha" moment for me was realizing that internal and external links are VERY similar, only a few parameters separate the two. While I knew that both were important, I previously (incorrectly) assumed there might have been a drastically different algorithm that handled both...

However, according to the documentation, it appears as if internal links and external links are more closely related than originally anticipated. Further reinforcing  the importance of good internal linking practices.

Within the links section, we also see that Google has a specific flag for when links come from a domain flagged as an expired domain. (This might be something they use for link penalties. If you accumulate too many links from expired domains, it might land you in some trouble).

In addition, they have a record of deleted links. This explains why we can often see the "ghost link" phenomenon in which webpages will continue to rank EVEN after links are removed. It's entirely possible that Google continues to count some of the links within their algorithm for some time even after the deletionDate.

(I speculate this is used to improve search engine result stability as link can sometimes migrate from the homepage, to inner pages and then move deeper into a site as content is shuffled around.)

Locality is interesting because they LITERALLY say that they are looking for links within the same BUCKET. This is as close as it gets to saying that those links are going to be worth considerably more.

And finally, parallel links indicates that additional links from the same domain might not count for as much.

Here's more information on the topic of links providing specific insights into how Google evaluates links.

Once again, we have a mention of the keyword bucket. This means that Google notices the set of pages / entities is the link associated with. We can assume that links within relevant content will be worth more.

In addition, we have "setiPagerankWeight" which is means that Google likely has a temporary value for Pagerank when Pagerank isn't yet calculated. (In order for the algorithm to work, they probably need a default PageRank value otherwise the algorithm might break. This is why brand new pages with no apparent links can still provide some temporary benefit as all pages have a default, minimum value.)

We start off with the "isNoFollow" tag which clearly indicates we are in the link section of the API. And to my surprise, the description reveals a small and subtle secret:

If a page has multiple links and just ONE of them is 'follow'  then ALL of them will be be considered follow.
(Note: There is no 'do-follow' tag. A link to be considered as follow by the absence of a no-follow tag.)

The next available API is TopicalWeight which, once again, indicates they are measuring link relevance! This is very likely a numerical value measuring how relevant a link is to the target URL, likely based on the content.

Another section of the leak describes the various attributes a link can have. This provides us with additional insights and confirms some existing theories.

From the description of additionalInfo, we can see that links from news hubs are noted. In other words: "Is a topic trending in the news" and receiving news links. I personally believe that Google treats websites that are trending in the news slightly differently than sites that have no news links.

Then we have another indication of clusters, which serves to identify the cluster that the link belongs to.

Finally, Google pays special attention to homepage links because historically, those were highly abused. In the description they elaborate that if they find a homepage link, they will verify to see if they trust the homepage.

Of course, we have indications that Google uses pagetags to identify page and relate them to the link. Nothing new here. 

To my surprise, we also see that Google STILL uses Pagerank within it's algorithm. While this was widely acknowledged earlier on in Google's history, they have since mentioned that they no longer use the original Pagerank.

I suppose that's technically true as they appear to use a new version of it called PagerankNS.

The NS part of the PageRank stands for "Nearest Seed" which is incredibly important. This will significantly impact how we evaluate sites.

Finally, as expected, links have a "Spamscore" associated with it which is likely derived by looking at the link text and surrounding text.

This part is incredibly important and I believe has been overlooked by a large portion of the SEO community. This is the first mention of NSR and has a huge multiplier attached to it to amplify it's effect within the algorithm.

While Google doesn't explicitly disclose what NSR stands for in the documentation, after analyzing hints from dozens of descriptions, brainstorming dozens of different possibilities and weighing the most likely answer, we can conclude that NSR likely stands for Normalized Site Rank.

Another link section includes an interesting API called "score".

With regards to the score, I believe they might be referring to the anchor text itself (as in, how likely is the anchor text to appear inside the text that it is linking to).

Essentially, this could prevent misalignment with regards to anchor text. So for example, if you were to create anchor text links for "cell phone cases" but you pointed them to a page about "skin care", then the score would be extremely low as it is very unlikely to appear naturally within the content.

Bottom line: We want links from relevant content, with relevant anchor text, pointing back to our documents.

One interesting API reference is the "boostSourceBlocker" which essentially means that a page can potentially be a valuable source for links while not ranking itself.

Historically, SEO professionals (myself included) might been wary of receiving links from those pages that weren't ranking... however, according to this API reference, in certain cases, we might be missing out.

It would make sense that links from directories, category pages, rss feeds and historical records could potentially be valuable even if they don't show up in search.

With regards to the other two API references, I believe that Google identifies the location of the link, making a distinction between links within the main body of a document (Main content) and links outside the main content (header, sidebar, footer, etc).

If this is the case, then it might be reasonable to assume that that links from main content are worth more.

Because "mentions" might be play a role in SEO, I thought it might be appropriate to cover it here.

The first API reference is: "AnnotationConfidence". I believe this might be the confidence in the mentions / annotations (citations/business name mentions) of the business. We've known for a while that citations can play a role with local SEO however I haven't seen as much testing with just mentioning the business name.

This might play a role and would be interesting to test.

In addition, I never knew that Google checks the relevance of the website linked within a Google listing.  This makes sense to prevent spam as we wouldn't want a local daycare Google listing to link to a supplement website.

I wasn't expecting this much documentation on anchor texts however it appears as if Google uses them extensively when evaluating pages and entire websites.

The first notable discovery is that Google combines external and internal text to determine anchor relevance.

In addition, they create a score out of many different metrics: 

- How many different anchor texts a page has.

- How frequently do we see the same anchor text, 

- How many external anchor texts are there, 

- How many internal anchor texts

And finally, it puts it all together to create a final anchor text score.

We also see that fragmented anchor texts (partial anchor text) still seems to count towards the main anchor text score... even if it's not an exact match.

This section highlights some of the more clever tricks that Google uses to rank a wide range pages online.

All these scores, from matchedScore, phrasesScore seem to be a calculation of the total anchor texts pointing to a site in order to determine the site's relevance. This re-emphasises the importance of anchor texts when ranking. 

What's really interesting is the solution that Google devised simultaneously avoid spam and while ranking queries for people that are searching for sensitive topics. (I'll describe how it works down below.)

Finally, we also see they have a trustedScore that verifies if the site is in the news (news links!). Here's how it can help you do better SEO...

Knowing that TrustedScore is determined by the anchor texts profile of the website means that...

Due to the importance of links and anchor texts, SEOs have been artificially creating them since the dawn of SEO. Here's how Google deals with link abuse.

Nearly every reference in here is important.

First, it confirms that Penguin updates periodically which means that if you are hit by a Penguin penalty, you'll have to wait until the next update to know if you recovered or not.

In addition, we can clearly see that one of the major components is counting anchor text. This indicates that they likely penalized webpages when too many repeating anchor text are detected.

We also see that pages can have a 'bad links' flag. While it isn't clear how a page gets this flag, I speculate that it might be manually assigned. In the early days of Penguin, Google used a lot of manual reviewers.

Finally, they check homepage links between sites hosted on the same server. For instance, if you have 10 or more sites all on the same host, and all the homepages link to a central location, they'll likely penalize the site for link abuse.

Here we see that Penguin is looking specifically at the external anchor text links. While we haven't seen any benefits to repeating the same internal anchor text link over and over again, internal links seem to be sparred from any Penguin penalty.

Another interesting reference is that there is a mechanism by which a page can be "protected" by early, trusted anchors texts. This was likely put in place to shield against negative SEO.

Finally, it's interesting that there is actually an anchor text limit! You can have too many links from different sources. For example, if a page receives 500,000 links (don't laugh, it happens), then Google will not calculate the score.

When it comes to backlink abuse, Google really hates homepage links from PBNs. They have a dedicated section checking for PBN homepage links. If most of your incoming links come from homepages... then you're in trouble.

We see that there's a special trust factor for anchors with PR5+ of power. This is interesting and lines up with what I have personally experienced in the past: Having one single link from a high PR page with relevant anchor text makes a huge difference.

In addition, we see that Google pays attention to anchors coming from pages marked with "expired". The mechanism behind this is that Google likely applies a flag to an entire domain if they notice that an expired domain has been revived with the same content. Then all the outgoing links from that domain will be marked with "expired" for a certain period of time. 

As there are surely legitimate cases where someone might re-use an old expired domain, I speculate there must be some time limit associated with the expired flag.

And finally, as we have previously seen, Google does have limits for the quantity of incoming link. Here they clarify that you can have a maximum of 5000 anchors pointing to a page. After that, they might ignore them or perhaps assign a penalty.

When it comes to link building, Google calculates the quantity of redundant anchors texts. (Repeating the same anchor likely gets you into trouble.)

They also look at the total quantity of domains linking, go too far and it can set off red flags.

And of course, they look at how many anchors per domain you have. (ie: In other words, avoid site wide links from another domain as it will surely set off a red flag within the Penguin algorithm.)

I was pleasantly surprised to discover a section dedicated to protecting websites from negative SEO attacks.
(It appears as if Google cares about SEOs)

It's very cool to see this: It appears as if there's an active "spam shield" that is activated when it detects a negative SEO attack.

If an abnormal quantity of links begins to point towards a specific page, it will temporarily demote the page while the attack is underway. Once the attack is over, it preserves the good anchor text links (built before the attack) while eliminating the bad ones created during the attack phase. 

This approach is clever because it makes the negative SEO attacker think their techniques are working, as they might see a temporary demotion of their target. Then, once they stop, the page magically bounces back to where it was.

This might be one of the most important sections of the entire API reference repository, as it explains how Google evaluates entire websites. Site quality dictates how all the pages on a website will rank; thus, understanding the site quality algorithm is critical for good SEO practices. 

Furthermore, if you've been impacted by a recent penalty, the site's quality is likely the root cause.

The first API reference discusses user generated content which is likely used to evaluate the effort that goes into forum threads, comments and possibly sites like Reddit. What is interesting here is that they are measuring the "effort" of the discussion. 

I suspect that the page might get a boost if the discussion is deemed high effort.

Next, we a have "Product Review site" promotion or demotion used to identify and rate product reviews. (Secretly, we all know this is demotion as it has been difficult to rank product reviews unless you're on a major authority website.)

Last, we have an "Exact match domain demotion" which was likely introduced to prevent sites like: best-drone-reviews.com from ranking. The act of registering exact match domains for ranking purposes was quite popular back a decade ago... these days, I personally recommend creating a memorable brand name for you

Now we get to the "site wide" elements that can make an entire site tank.

We have "nsrConfidence" which is the confidence in the "Normalized Site Rank" score. As we previously covered, the Normalized Site Score is most likely a measure of how the site performs compared to the rest of the industry. I believe it is one of the most important metrics and this "nsrConfidence" evaluates how trustworthy the score is.

Then we have "lowQuality" which is likely a flag for a bad site. We see that it is pulled from the NSR data which means that when the normalized site rank is too low, the site receives a "low quality" flag... and then likely doesn't rank at all. If you've seen websites remain in the index but refuse to rank, then this is probably the reason.

The "navDemotion" is likely a demotion related to NavBoost, perhaps calculating how much it should drop a site in rankings. 

And of course... *drumroll* they have a measure of site authority in the form of "siteAuthority". 

This is notable for two reasons:

1. Google has denied having a site authority metric in the past. They LITERALLY have an API reference called SiteAuthority so I don't feel as if the Google spokesperson was being straightforward.

2. Once again, we see that the "site authority" metric is derived from the normalized site rank! If you weren't already convinced, hopefully you can see how important normalized site rank is within the entire algorithm.

I knew that they had worked on revisions of Panda a few years ago as I was involved in recovering websites affected by Panda (I created case studies on how to systematically recover websites from Panda)

However, I wrongfully assumed that they had simply updated the existing Panda algorithm...  It seems as if introduced a NEW panda algorithm called: "Baby panda". Surprisingly, it seems as if this new baby Panda (user experience related algorithm) is applied ON TOP of the original Panda? 

Yikes! This means you can potentially be penalized twice for poor user experience.

Conversely, there also seems to be a boost for 'authority sites' so perhaps if your user signals are great, it go the other way. And of course, a demotion if your anchor text links don't match the content.

Finally, raw click signals are used to evaluate the site's performance as well. Click signals (from Chrome) are incredibly important for determining how a site performs in search.

We see that the topics of the site can be accessed by the algorithm. This makes sense as Google needs to be able to retrieve sites that are topically relevant to the query.

Then we have a measure of "Scamness" for a website. While we don't know much about this API reference, we can infer that they are using AI to measure it. 

And finally, an unauthoritative score. Perhaps this is based on links, user experience or originality of content.

Here we find the original Panda!!

While the Google algorithm has changed significantly since then, Google is still focused on the overall user experience... albeilt they are measuring it in slightly different ways.

The recent Helpful Core Update, March Core update and more are all focus on the user... and they are likely using additional signals (ie: from clicks) to measure it.

It does seem somewhat unfair that all of these penalties stack on top of each other. 

From unauthoritative score, to click signals, to normalized site rank, to Panda, and even BabyPanda—if you're affected by one, then you're likely to be affected by all of them. This is the main reason why some websites impacted by one algorithm change notice a subsequent drop after the next update, and so forth.

I found a section that discussed an explicit "siteScore" and dove into what that could possibly entail.

First, I think it's notable because just like site authority, Google has also mentioned there is no site score...

Well, there is.

Interestingly, they don't use the entire site to calculate the site score. Instead, they note the percentage of the site used to estimate the score. This is likely to save resources.

Here we have a section that many SEOs will be interested in... topical authority.

When Google refers to embeddings in the context of search, it means transforming words and phrases from web content into vector representations. These vectors help Google's algorithms understand and quantify the relationships and relevance among different textual entities, enhancing the accuracy of search results.

In plain English, it's like Google creating a digital map of all the words and phrases found on the website.

We see with the "SiteEmbedding", "SiteFocusScore" and "SiteRadius" that Google looks at both the page embedding AND the site embedding to determine the topic. This means that other content on your site will dictate how well you rank.

In addition, it also measures how focused on a topic a site is... very likely providing a significant ranking boost for sites that have a narrow focus.

And finally, it will measure how 'unrelated' a page is to the rest of the site. Creating unrelated content on a website will likely not rank as well.

One of the my favorite areas of SEO, on-page optimization! In this section of the API reference documentation, we dive into how Google looks at content.

The first thing we notice is that they are looking at both the entities AND the semantic node in which this document lies when evaluating content.  In other words, if this document is part of a related, relevant set of documents. 

This is just another reason to have topically aligned content.

Next, they are looking at the outgoing links in a document. While we had already tested for this (pages with relevant outgoing links ranked better than pages without any links), it's nice to confirm that they are monitoring it.

And of course, they are looking for fresh & updated content, as noted by "lastSignificantUpdate".

Google knows the difference between minor updates and significant updates.

Once again, we see that Google sort documents by using entities.

Entities are specific words or phrases that are recognized as representing distinct and well-defined concepts or objects, each carrying an associated meaning based on real-world references.

For example, the word "party" can have multiple meanings.

1. A party of 5 people
2. A political party
3. Let's go to the party!
4. I like to party with friends

While the word stays the same, the entity identifies which one of these categories the word falls into. Within a machine learning context, it is very important to make a distinction between a political party... and politicians having a party! (And that's why Google uses entities to classify and rank all documents on the web)

Closely related to entities are topical categories, which they mention when discussing "topics".

Surprisingly, they can also flag "Golden" documents that human reviewers deem important or as a gold standard. I'm not sure to which capacity the golden flag is used however it would surely give the document an unfair advantage over all documents.

Here we see that when Google is analyzing a document, they try to identify ONE main focus entity. I'll sometimes refer to this as main keyword of a document even though it should probably technically be called the "focus entity" (somehow that just doesn't have the same ring to it).

An interesting bit is that they also note if a date is mentioned in the title / URL, then they check if this aligns with the other dates found within the document.
 A while ago, a Google engineer mentioned that you should avoid just updating the date in the title without updating any other information on the page... this is likely why.

Finally, there's a special note for checking if the page contains private information. (For example, a person's home address, credit card, maybe social security number or phone number.) In my experience, when Google finds private and sensitive information, the page is less likely to rank. <cough> negative seo tactic</cough>

Within this section of the API reference material, Google shows us the information they store about webpages.

Apparently they have reference pages for entities... like Wikipedia!

That's cool... because it means if you want to be highly relevant for an entity, you might be able to look up the Wikipedia page associated with the entity and include similar terms.

In addition, there's an API to list all the links with scores associate with a page! (This would be cool to use. Imagine Ahrefs site explorer... but with Google's own data. It would be incredible to see the actual scores associated with each link built to a page. This would make link building SO much easier.)

We can also confirm that links are relationships between entities. We already knew this however it re-confirms that you want to get links from related content.

We also see that Google makes a note of all the names associated with page. This is likely very useful when deciding which documents to retrieve from the index.

We continue digging into the page information and see they have a section for original content.

Personally, I believe this refers to original content vs duplicate content.

And not, 'how original' content is.

Google will likely not rank you if you have a bad SSL certificate.

Google cares a LOT about the recent creation and expiration date of a domain. In this section they elaborate (not shown) on something called a "DomainEdge signal" that I suspect is likely used to fight PBNs.

It's interesting to see that every webpage has a rich snippet (even if it's not shown). More on how Google evaluates rich snippets later on.

Interestingly, they store a list the related pages to a document which is likely determined by user behavior. Within patents, they describe that they create associations between subsequent user searches. 

Perhaps if you search for document X and THEN search for document Y, an association is created between X and Y.

Another small discovery within the document information section is that Google has an option to NOT show a page if it's Pagerank is lower than a pre-determined amount.

This section dives deep into content, penalties and spam. Our aim is to understand what constitutes great optimized content while avoiding over-optimization

Inside the documentation, there are quite a few spam flags. The first, "uacSpamScore" might stand for User Automation or User Activity.

The next "Spam tokens content score" & "trendspamScore" references suggests that Google might have a list of spammy words they use to measure spam. Perhaps many mentions of casino / viagra might trigger it.

Within the date section, Google mentions a "FreshnessTwiddler" (Twiddlers are modifiers used by Google, usually to boost rankings). In spite of what Google has claimed in the past, this is very likely a freshness ranking boost given to fresh content.

In addition, we see that they do, in fact, have a "YourMoneyYourLife" score. Whenever you're posting content on the web, Google is checking to see if it falls within this category and if it does, there might be an additional layer of verifications and/or requirements.

The next API reference is incredibly important:

The 'TopPetacatTaxID' API reference, in my opinion, indicates that topical relevance is very important when it comes to Google ranking. This suggests that Google is classifying websites and assigning ONE MAIN category to them. 

It is likely that this categorization is used throughout Google's algorithm, influencing both content and link building.

The bottom line is that content aligned with the site's primary topic receives a ranking boost. For example, imagine a user searching for 'best dog food for puppies.' The 'SiteboostTwiddler' would analyze the query, recognize that it pertains to pet food, and then use the 'TopPetacatTaxID' to prioritize results from the top category of pet food.

Continuing with the content, we see that they have an explicit mention of: "OriginalContentScore".

Although it might still be a ratio of duplicate content versus original content... I believe that this API reference is measuring how original the content is versus your competitors. Google has long encouraged publishers to create new, original content and this seems to be an attempt to measure that effort.

I believe that if you're just creating an carbon copy of the existing search engine results, there is no real incentive for Google to rank you above the rest. That's why I encourage webmasters to go above and beyond the #1 result, using entities that your competitors aren't currently using and adding unique information on your subject.

Once again, we see another freshness signal that indicates that Google is rewarding new and recently updated content.

And...

There's spam score if your name is Eric? That's not nice 🙁

Now here's something I didn't expect:

To my surprise, they seem to have "Biasing data" entry. This might be a signal that measure how neutral or bias the content is. ie: overly salesy affiliate pages might not rank as well.

And finally, spamCookbookAction hints that there are a specific set of rules that trigger spam on pages. For example, invisible text and other shadier tactics might fall into this category.

As we continue looking into the content documentation...

We see even more indications that topical relevance plays a significant role in ranking. Whenever Google speaks of clusters (the relationship between local pages), they are denoting the topical content. The idea is that if you're an expert on a specific cluster, then you will likely rank better when there are queries within that cluster.

Next, they have a specific keyword stuffing score! We've long known that Google prohibits keywords stuffing so it's nice to see it here in the flesh. Even today, I still see instances of webmasters keyword stuffing their titles (don't do this).

Finally, the spambrainTotalDocSpamScore is an indication that they also use AI to estimate the document spam score. Whenever Google mentions "brain" in an algorithm, it is their way of saying they are using machine learning to accomplish the task. This implies that they have trained a machine learning algorithm on a slew of spammy documents and then ask the AI to classify your document's spam level.

We don't know exactly what kind of training data Google originally provided for the AI spam algorithm however it's logical to assume that if your document looks like a spammy document, then it will likely be classified as one.

As we continue digging deeper into the data used within content classification, we discover that "spamrank" is a measure of the likelihood that document links to spam.

Suffice to say, if you link to bad places within your content, this will hurt your rankings.

This also means we have to watch out for accidental links within our content. Sometimes there are malicious actors that will try to add hidden links within content, sometimes people will add redirects after a link is placed and sometimes we can make typos in our URLs leading to incorrect placements.

QualitySignals & Crowingdata both likely represent user signals. Google cares a lot about user signals, to the point of saying that "they don't understand content, they fake it".

New websites have a sandbox period to prevent spam.

We've long know that there was a sandbox period for new websites however some Google employees denied this in the past. Now we see that new websites in fact DO have a sandbox period to prevent spam. It is determined by hostAge.

As we continue on our journey into Google's spam filters for content, there are a few more interesting tid-bits:

First, we have "Gibberish score". This is likely to address lorem ipsum testing and/or just random characters on a page. (Anecdotally, this must not be the strongest ranking signal as I still have pages ranking with pure 0000 and entities which should, in theory, trigger a relatively high gibberish score.) Still, it's nice to know that it at least plays some role in ranking.

Next, we have fresh again... however this freshness might be measured on various different levels. For example, freshness at the host level, blog level and finally, page level. Perhaps it's not enough to have a single fresh article if the website hasn't been updated in ages? (That said, if an article is updated, then wouldn't the site also be updated?)

And last, this is a really cool discovery: onsiteProminence: Google tries to determine how important a page is on a website. They determine it by computing from SIMULATED TRAFFIC FROM HOMEPAGE!

Another small discovery is that Google measures the commerciality of a page. This implies that pages selling a product might not rank as well for certain queries if that query is not a commercial term. I suspect that Google would also analyze the incoming query to determine the intent:

If a user's search term indicates they are seeking to purchase products, then results with a higher commercial score may be shown. Conversely, if the search term only seeks information, then pages with a high commercialscore might be avoided.

Next, we also have a SpamWordScore which implies that Google is looking for a specific set of words on a page.

We also seem "SpamBrainDomainSiteChunkData" which indicates that Google is using AI to process and identify spammy domains. Across all the references that we have seen, it appears as if Google is using a combination of fixed word seeking and artificial intelligence to determine spammy content.

When Google is evaluating content on a website, it looks at the hostNSR which very likely stands for Host Normalized Site Rank.

Here's how I believe it works:

Next, we have our first mention of "Firefly". This is an important signal based on the frequency of posting / how many people click on the new articles.

Let's dive into the ramifications of this signal.

Firefly seems to be a major ranking factor that measures how users react to new content being published on the website.

Here's a quick tentative overview of the process:

1. New content published
2. Google artificially boosts the content to users to see how they interact with it.
3. Google measures interaction and calculates score for site.

We jump into a section called Keto score which is likely a power prediction made so that Google can rank content without fully processing a final link score. This allows Google to quickly rank pages (in minutes) for breaking stories without having to go through extensive link scoring (this is done later and then the rank is re-adjusted).

First, when it comes to predictions, it appears as if Google uses AI to estimate the effort that went into creating the page. Crazy! (This can likely be manipulated by writing about the effort spent in the introduction.)

The rest is likely related to the power predictions based on existing signals for the website. I believe that the final "keto" score is the default predictor value that is used for all brand new content on the site.

Here we jump into another content-specific section of the API documentation.

First, the "ugcScore" most likely stands for "User Generated Content score" which measures the quality of user generated content. I believe that this is a critical metric to calculate because when Google wants to show discussions from Reddit, it needs to be able to distinguish between good and bad threads. 

It most likely tries to identify useful information, how many people are discussing the topic and so forth.

Then we have an API entry called: "Title match score" which measures how relevant the queries are to the titles. You obviously want the title to be relevant to the query... however I speculate you MIGHT be able to go too far if the titles ALWAYS match the exact query. Further testing is required in order to confirm if this is used for promoting relevant content or if it's used to penalize overly optimized websites.

And now for the API call that seems to have been the focus of many discussions: SmallPersonalSite.

First, SmallPersonalSite is a BOOST for very small sites. (It literally says promotion in the description)

When websites have very few pages, they receive a ranking boost in order to be able to compete with larger sites. This initial boost goes away once your site reaches a certain size.

Google put this in place specifically to help the tiny mom & pop websites that are published without any regard to SEO. It helps professionals with single page websites be found, it helps the tiny 2 page businesses be found, etc.

Without it, people would struggle to rank for their own name.

As we continue in the content evaluation section, we see a mention of "clusterUplift". This is likely a boost for content within a topically aligned cluster. Having related content is essential for ranking on Google these days and this is just another piece of evidence that explains why this is the case.

Next, we have "siteAutopilotScore". This might be a measure of how much a site is created by automated processes: Auto-generated content from RSS feeds.

Finally, we have "chromeInTotal" which is very obviously the total Chrome views a site has received. I suspect that that the quantity of Chrome views for a site might be a ranking signal... however it certainly wouldn't protect sites from being penalize.

Chard seems to be related to Chrome interactions.

Even though there is no direct mention of what "chard" stands for, I suspect it is somehow related to user behavior and Chrome.

However, in the context of data processing and distributed computing, "shards" are commonly known. Sharding is a database architecture pattern related to the practice of dividing a database into smaller, more manageable pieces called "shards."

To me, it sounds as if Google combined both Shards and Chrome together, resulting in Chard.

A previous guess was that it stood for: Chrome Average Duration. 

I personally believe they are collecting user data from multiple places and this produces a user score.  Perhaps that includes how long people stay on the site using Chrome.

Whatever it is, this seems to be a VERY important signal.

As I was browing the site evaluation section, I stumbled upon this API reference that at first glance seemed harmless... however as I discovered the impact of NSR on search results, it became quite apparent that it was immensely powerful.

Here we have "nsrOverrideBid" which means that it can be used to OVERRIDE the Normalized Site Rank assigned to a site. Essentially, what this means is that they can manually boost a site's rankings if they want... (in case of an emergency, of course)

In this section, we dive deeper into ranking (calculating a score) for content. While we've already covered many of the elements (ingredients) that Google takes into consideration when evaluating content, this section seems to be more focused on scoring.

First, we have a new discovery from Juan Gonzalez that spotted that AGC likely stands for "Artificially Generated Content" which means that Google likely has a score that measures how much artificially generated content is detected on a website. (This is why I believe it has been a good idea to avoid publishing generic ChatGPT content and instead, I have opted to use highly modified / stealthy AI content to reduce the likelyhood of Google detecting it.)

According to Wikipedia, Racter was a very early artificial intelligence program by IBM. This helps supports the theory that AGC stands for "Artificially Generated Content". 

This score is then used within the larger NSR score (Normalized Site Rank) which seems to be the primary driver of site rankings.

Next, we have the article score that is determined by the classification of the site. This seems to indicate that if a site is related to the topic, you automatically get a boost to your score even before Google begins to analyze the content.

Then, we have Site2VecEmbedding which is is related to the topical alignment of the site. This, once again, indicates that Google is looking at all the entities on a site in order to determine the relevance of a website.

I thought some people might be interested in knowning that Google can manually flag sites that are authorities in elections (likely government sites). In an effort to fight misinformation, sites like Whitehouse.gov or local city election sites might be flagged so they appear higher when people are searching for recent election related information.

Finally, we have a clutter score. The "ClutterScore" is a score based on the layout clutter, used extensively on mobile when you have too many ads / pop-ups.

If you're running mobile ads, make sure that the ads aren't popping up too much on mobile devices as it will increase the clutter score (and this will affect your rankings).

While this applies to desktop as well, the clutterscore is a score for instrusive elements on a page.

Clean layouts will typically rank better.

As words can have multiple meanings, computers prefer to use entities to classify and understand content. In this section, we'll discover the emphasis that Google puts on entities.

First, we have a raw list of the entities within a document. Entities play a central role in what Google retrieves whenever there is a search query.

The next thing is looking at the relationship between entities. This can help discover related terms, related topics and help provide context for a document.

Not only does Google cares about the entities mentioned in documents, they also note the frequency and importance of each entity. Entities that are mentioned more frequently are going to be seen as central to the document and more important.

Another interesting point is that they share that "implicit" entities are calculated when you mention certain entities.

For example, if you mention the word "gym" in the content, then a related entity can be "sports". This isImplicit is VERY useful as it allows documents to be found even if they don't mention the exact entity.

You can try it be searching for: "motion picture about a tsunami" on Google. You'll likely see results about movies featuring a tsunami... even if the words "motion picture" does NOT appear on the page.

This is because Google knows that the motion picture entity is related to the movie entity. (And this is why you don't need to mention every single entity variation in order to rank.)

Of course, Google will not show the page just because it features an entity. The confidenceScore indicates that Google is seeking the most important entities in a document.

Continued here, we see they list all the most important entities in reverse order: From most important to least important.

And as we previously covered, they list latent entities are going to be the related words associated with the main entity. We can think of these as related words to the main subject.

This is useful because Google already knows the related entities to the main entity... and therefore Google can seek pages that contain those latent entities.

So can you just throw a bunch of entities on a page and hope for the best? Well... kinda (it works). However there's a bit more refinement to be done in order to get the absolute best rankings.

Here we see a mention of IDF which stands for "Inverse Document Frequency". This is indicate that Google is measuring original content. (Mention things/entities no one else is discussing). To do so, they analyze how rare an entity is with regards to a corpus of text... if it's something that isn't commonly mentioned, then you're likely writing about something original.

Next, we have mentions of "Salience" which is a measure importance. Google is always asking itself: What is the most important entity in the document.

At the higher level, beyond entities is categorization. Which categories does the content fall into?

According to the documentation, it appears as if Google classifies content into categories.

In appears as if they have recently upgraded categorization and currently have two systems running (and old and one new). I speculate that the old one might be the list of categories that they published at https://cloud.google.com/natural-language/docs/categories as it was a comprehensive list of NLP categories.

One of Google's biggest advantages in search is its unique access to user data. Google monitors how users interact with content to make informed judgments about that content.

This section of API references provides us with many clues about how they measure user clickthrough rate.

First, the obvious: They measure clickthrough rate (impressions / clicks) for content.

Next, we also note that they have a "bad clicks" (likely people returning to search)

And finally, unicorn clicks which is clicks from abnormal users trying to manipulate rankings. (ie: a user clicks on 1000 results a day)

We just looked at someo of the API references for the clicks themselves however there's another section with the accumulated click data.

This is SUPER important because it is the ACCUMULATED click data for a website that is likely responsible for penalties. If you've been experienced a recent demotion, it's likely because the data here for your site is sub-par.

Most of it is what we'd expect, good clicks, bad clicks, etc...

However small surprises are:

1. Image clicks count. I imagine that these are images from within the normal search engine results however could it also count information from the separate image search? We'll have to test this.

2. Dwells count. This means that even if you don't get a click, if people hover the search snippet, then you get rewarded. This is a nice addition from Google to reward high ranking sites that currently have the search snippet.

Another interesting part is that they measure the last time a document received a click. Perhaps this measures the popularity of content within your site as we can assume that if content receive no clicks, then it might not be as relevant. 

The next important part is the mention of "LocalUniversal" dwells. While we have no additional on what the "LocalUniversal" source is... perhaps this could be an indication they are measuring user data from OTHER sources such as Android.

This is yet another section clicks and user enagement on a website... with all this data being used to boost search results that perform well and demote content that doesn't.

The first thing we notice is the extensive mention of CRAPSdata. This data, gathered via Chrome and mobile devices, is apparently very important.

Unfortunately, Google doesn't share exactly what CRAPS stands for. However, an educated guess might be:

"Chrome Requests Aggregated Page Statistics"

Update, apparently according to an ex-Google employee, it might stand for: "Clicks and Results Prediction System."

The bottom line is: If users visit and like your site, you will do well on Google.

As we know by now, they measure the query and how users react to it. The idea is that every click counts as a vote . The more votes a document receives, the better it likely is.

In this section, they discuss how Navboost is used in combination with clicks to help improve Google's search results.

Because we know that Navboost is a method of increasing the rank of pages... and that Google is trying to measure clicks, we can deduce that Google temporarily increases the ranking of pages in order to measure clicks from it. 

The more "longclicks" a page receives when it is boosted, the better it is. This could help surface pages that otherwise wouldn't be found.

Finally, we have a mention of "aggregatedQueryCount"at the end of the documentation. This might be a ranking factor on it's own... how many times this document appeared in search queries.

More popular pages might appear in more queries, especially if this page covers many topics (might help pages that rank for many different topics)

Speaking of mobile, Google an entire section dedicated to mobile related verifications and penalties.

Bottom line is, in order to rank, it is very important that the page be "Smart Phone optimized" which essentially means "Mobile-ready" in Google speak. You can (and probably should) check the Google search console for your website to see if any pages have issues. When you're in the search console, it's within the "Page Experience" tab.

In addition, too much advertisements on smart phones will likely result in a demotion/penalty.

Interestingly, we have a section dedicated to the search snippet.

The algorithm surrounding the search snippet seems to fairly straightforward:

- There's a title score that measures the relationship between the title and the query
- There's an embed score for the potential passage and an embed score for the query 

And I suspect the radishScore at the end might be putting everything together. Perhaps it stands for something like: Relevance, Authority, Density, Intent, Specificity, Helpfulness

In which, Google would seek relevant content, from authoritative websites, with a high density of entities, that matches the user intent and provides specific data that is helpful to the user.

And finally, we have document retrieval which is obviously very important for search.

As we have previously seen, factors that determine which content Google will retrieve when there's a query:

- Last page update. Google seems to want to surface freshly updated, new content.
- Queries with a Navboost
- And of course, PagerankNS (the new pagerank).

Google lists all the terms that a page should rank for and the category in which is belongs.

This is likely what they use to retrieve documents before sorting them to rank them.

And that's a look at all the most important API references that can help us learn about the Google algorithm.

Putting it all together, here's my new ranking process combining all this newfound knowledge. 

This is designed to take advantage of all the newly discovered "boosts" that can help a website dominate the search results while following all the rules. I distilled it down to be as concise as possible in order to make it simple and easy to understand.

Please note that this is my own personal ranking formula. I have no control over you, your site, nor Google, you are responsible for any changes you make to your website.

When I'm starting a new site, I'm purposely going to keep it SMALL at first in order to take advantage of the "Smallpersonalsite" ranking boost.

I will have a homepage and 2-3 other pages so I can quickly gain traction in the search engine results.

Taking into account that there might be a sandbox period dependant on hostAge, I will be patient even if I don't see instant ranking results.

I will have ONE flagship article that is immensely valuable. I will aim to get it shared across social media and get natural visitors to my site.

This will, in turn,

1. Increase Chrome views
2. Improve my overall site score
3. Improve my click metrics / visit duration

Essentially, setting up my site for success as all the Site Quality metrics will be positive.

Once I'm confident my site quality is set, I'm getting visitors and a I have a few links, I will leap out of the "small" site size and start adding more content.

I will not be discouraged when I notice a small drop in rankings as I make this transition because I know it's temporary as I will eventually surpass my small size.

The content I'm adding will all be HIGHLY related and topically aligned. I will aim to keep the focus of my site very narrow to take advantage of the second boost.

I will focused on writing on content that falls into a specific category and is semantically related.

For example, I will create a topical authority map of the terms I want to cover.

To build my topical map, I will look up:

- Related entities to my main term
- If applicable, I will look up a central wikipedia page on my entity to retreive semantically related entities.
- I can also look at the knowledge graph for ideas and hints on semantically related entities.

I'll gather a list of approximately 20-50 semantically related topics to cover. These will branch out from my main focus entity and will be inter-linked together.

When writing, I will use as many related entities as possible and make it clear what my main "focus entity" is.

Entities (Analyzed With Google's NLP Model)
https://on-page.ai

I will ALSO make sure to include entities that my competitors aren't using so that my content is seen as original (and not just a copy of what's already there).

After creating content, I will make sure it receives Chrome views by sharing it on social media.

If an article is not receiving Chrome views, then it's probably not good enough and it will struggle to rank.

I will be putting out regular press releases about my site as Google seems to notice when links are coming from news outlets.

However, I will be VERY wary of pointing my press releases directly at my content as link anchor text plays a large role in ranking.

Therefore, I will have a "news" section on my site where I point the press releases.

Google puts a significant emphasis on anchor text when ranking and internal links are ideal for creating anchor text relevance.

Whenever I create new content, I will also create relevant internal links that use highly descriptive anchor texts.

Specifically, I will use the exact match anchor text once and will vary it slightly afterwards. (I don't want to go overboard with the anchor however I do want to keep it related).

I will seek relevant links from related content pointing to my site.

Specifically, I will try to get links from seed sites (or more realistically, major sites that have links from seed sites.)

I will aim to publish a minimum  of ONE high piece of content per month in order for Google to keep my active boost and this will let them see how users react to my newly published content.

(Also not previously mentioned, I will use the same consistent author as Google makes a note of the author used.)

I will be very weary of getting links from sites that might have an "expired" tag. If I'm getting links, I will try to get some from domains that haven't dropped. 


I will also be very careful with anchor text and link velocity. Too many links with exact anchors will likely trigger a penalty. Same goes for homepage links. 


I will also avoid using click-services as most users are likely going to be flagged as "unicorns" so IF I do send fake traffic, it's going to be from a unique, low volume source. (Maybe a home made set of phones).

One of the biggest realizations is that there's a lot more than just the words on the page that go into ranking a single page.

The words / entities are immensely important HOWEVER it's also the entities found on other pages of your site that will affect the rankings of your main page. (Topical authority).

And it's also how users interact with your entire site that will affect how that pages ranks.

Finally, it's also the internal links and the external links anchors that will dictate rankings.

Mastering Google traffic can be challenging at first however once you do it... you have access to the lion’s share of traffic for your industry.

This project represents a significant investment—over 158 combined man-hours from my team (including 126 from me) and numerous late-night sessions as I was pushing hard to complete this report for you. This is all I did during the month of June.

The reason I have chosen to make this publicly available at no cost is to help as many people as possible that have been recently affected by Google updates. I believe the SEO community has been pushed around enough and it's finally time to fight back against unfair practices.

To those people, I hope that by sharing some of my work helps you.

If you run an SEO agency, I hope that some of my theories, data and tests help you better serve your clients. You can be the hero that rescues a struggling website.

If you're an SEO professional, then the data might make more informed decisions, saving you time, money and effort.

Should you feel this resources valuable, feel free to share this page within your professional network, within Facebook groups, Slack/Skype/Discord chats and forums.

And should you wish to further support me in any way, I invite you to check out On-Page.ai. Created with Google's best practices at its foundation, the platform aims to optimize your rankings by staying ahead of Google's frequent updates. One of my main motivations for keeping up with the latest SEO trends is to continually refine On-Page.ai, ensuring it embodies the most effective ranking strategies. From discovering the most relevant entities to link building with relevance, to building topical authority, and creating incredibly well-researched content, On-Page.ai has you covered.

I'm confident that it can be a valuable asset in ranking your website online.

Here are some useful links:

- Google API References

- On-Page.ai

Should you wish to contribute any discoveries, case studies or any corrections, please email me at team@on-page.ai

Sincerely,

- Eric Lancheres

Eric Lancheres

Eric Lancheres is one of the most respected SEO experts in the world. He has been featured as an authority at Traffic & Conversion Summit, SEO Rock Stars, SEO Video Show and countless other interviews and podcasts. 

For 12 years, he researched SEO trends, coached business owners and agencies, conducted experiments, and published his findings.

Eric has poured all his accumulated knowledge, best-kept secrets, and proven processes into the development of On-Page.ai, making it the go-to tool for effective online ranking.

© Copyright 2024 On-Page.  All Rights Reserved