Introducing: Google’s algorithm

Algorithms sound like an incomprehensible concept, but can be simply defined as a set of rules for solving a problem or accomplishing a task. They range in complexity from very simple scripts to computer-driven methods beyond human comprehension.

Here are some examples:

Simple Task:
Washing clothes

Goal: Get the clothes as clean as possible without shrinking them

Algorithm to accomplish the task:

1. When I wash delicate clothing, set the washing machine to Cold
2. When I’m washing towels and regular items, set it to Warm

Less Simple Task:
Cooking a full course meal

Goal: Serve four different dishes at the correct temperature at the same time

Algorithm to accomplish the task:

1. 1. Pick a serving time for the meal
   2. Read the packaging on any frozen items 
      • Check how long the recipe specifies to cook them
      • Set them out to thaw in time to go into the oven
   3. Preheat the oven
   4. Start preparing the cold items like salads
   5. Prepare any items that will be cooked in the oven 
   6. When the oven reaches the proper temperature, put in oven-cooked items in order of longest cooking time to shortest cooking time

And so on…

Seemingly Impossible Task:
Generating search results for any query a person can come up with

Goal: When someone types in any combination of words, abbreviations, and numbers, deliver a full page of relevant web pages for any keyword(s) in less than a tenth of a second

Google’s problem happens to be a lot more complex than laundry or cooking. The list of rules and steps required are well beyond human comprehension. Their algorithm is actually built and modified on an ongoing basis using “machine learning.” Human beings fine tune the code that actually writes itself to become more effective over time. Rather than a simple list of steps, the Google algorithm is a flexible and ever changing computerized process. But, if we break down the steps of how Google fundamentally works, their algorithm is at least something a person can envision:

Step 1

Google needs to keep a copy of every page on every website on the entire internet

What sounds impossible is surprisingly doable with the power of computers. A program called Googlebot spends its days crawling the web and saving a copy of every page it visits to the index. 

Googlebot crawls the average small business website every 3-4 weeks, but SEO’s can use special tools to request additional crawls as needed. For example, if you added a new product or service page the day after Googlebot visited, it could take weeks for it to appear in the index (and search results) without a manual request. 

Mobile vs Desktop

A relatively new feature at Google is their use of Googlebot sending a “secondary crawler” to your site. In most cases, the primary crawler is looking at your website through the mobile version of Chrome and the secondary crawler views the desktop format. Any pages that don’t perform well on both formats tend to appear lower in search results, especially on the type of device that saw errors. If your website performs poorly on a mobile crawl, it will appear lower in mobile results than on desktop. 

Keeping Unwanted Pages Off Google

SEO’s can use code snippets called “robots meta directives” to specify which pages of a website should or should not be crawled. If you have a page on your site that you prefer not to show up in results, it can be omitted and Googlebot will keep it out of consideration. There are many different scenarios where meta directives come into play:

• On e-commerce sites, the “shopping cart” page is rarely a great place for a searcher to start. In most cases it is helpful to block Google from that page and instead have them show your homepage, shop page, or anything else that is packed full of useful content.
• If your site has multiple variations of a single page, it is a best practice to let Google know those pages are grouped together. If you offer one product in 4 sizes and 8 colors that could look to Googlebot like 32 different pages. In reality, it’s just one page with selectable options. SEO’s implement “canonicalization” in your website’s source code to keep Googlebot from mistaking versions of a page as duplicates. 
  • Duplicate content is a big concern for SEOs. Anytime Googlebot perceives multiple webpages as being nearly identical, it chooses one (typically the oldest) and refuses to include the others in search results. Properly canonicalized content shows Googlebot that you have different versions and informs it which one you want shown in results.
  • Canonicalization also helps consolidate link and quality signals to one URL!
  • In this scenario, a page like /products/champion-jacket/ is preferable in search results versus allowing Google to guess and potentially choose something like /products/champion-jacket/size-XXL/color-yellow/ when someone searches “medium black champion jacket”
• Another reason to direct Googlebot away from certain pages is if you have pages that are only for paid or logged in users and would otherwise appear blank. Including a meta directive will keep Google from including those pages in search results.

Step 2

Google needs to understand the basic purpose of every page in the index

The Google Webmasters site and other Google press releases alert the public to changing rules and methodology, but Google doesn’t actually publish what factors they use or how factors are weighted when it comes to picking which sites show up first. They are very cautious in opening up the door to manipulation. Scammers have long sought to outrank legitimate sites for the purposes of stealing credit card info and spreading misinformation. Google holds their cards close for that reason. 

Google discloses that they use hundreds of factors in evaluating what types of keywords a given web page is relevant to, but they do not publicize that list and instead encourage users to focus on providing them with great content on fast-loading and technically-sound websites. 

Step 3

Google needs to recognize the intent of each search query and deliver the results in order of relevance

After sorting through all of the pages across the Internet that could possibly meet the searcher’s needs, Google essentially has to guess which of those relevant pages are the best fit. To do this, they use the behavior of past searchers and little clues in the search term itself. 

Think about searches that include the word “apple.” Users might be looking for a new phone,  somewhere they can buy fruit, or a stock quote. How can Google tell the difference?

For each of these types of terms, Google has built on past experience. If you search for “apple price,” Google is quite confident you want the stock price so the results will start with that. However, the page quickly transitions to a handful of pages selling iPhones. You won’t find the cost per bushel of the fruit until somewhere around page 3 or 4. If the majority of “apple price” searchers started clicking all the way through to page 4 before visiting a page about fruit prices, that may soon change…

Step 4

User behavior is used to improve the algorithm

The way we interact with each search result page is used by Google to measure the quality of their results. They track our scrolling, clicks and time spent on different websites to measure how good of a candidate each result is for a given topic. Using that data, the algorithm constantly improves itself with minor tweaks.

For example, there are millions of pages on the internet dedicated to selling dresses. Most of those websites sell prom dresses. If the searcher types in “cheap prom dresses,” Google tries to predict which of those websites are the most suitable. If prior searchers looking for “cheap” or “inexpensive” dresses tended to hit the Back button quickly after visiting [site A], but spent quite a bit of time on [site B], Google will likely reshuffle those results to list the second site above the first site. Even though [site A] might have more dresses or a better website, Google’s ultimate goal is to send searchers to pages they will like.

Step 5

Google customizes the results for each user

Personalized results go beyond traditional ranking factors and further adjust the results using factors like:

1. Where the searcher is physically located
   • Businesses and content closer in proximity to the user will be given additional weight
2. The types of searches and web pages the user previously entered
   • Google will start to make assumptions about the type of topics you are likely to be interested in
3. Which results the user has recently clicked on
   • Google uses your history to decide which websites you like visiting and also which results do not seem to be answering your questions
4. What type of device the searcher is using
   • Search results change by as much as 30% between a computer and a mobile phone.
5. Information from other Apps such as Gmail, Drive, and Google Play
   • By reading your emails and files, Google can use complex ideas like the dates and location of your next vacation when deciding which results to show you