Search Engine Optimization Part 1: How Does It Work?

SEO is an essential aspect to website creation.

SEO Part 1 - What is SEO and How Do Search Engines Work?

What is SEO and Why?

SEO stands for Search Engine Optimization and includes the techniques or methods that increase the amount of traffic to a website. In the more modern age of SEO it is not solely about search engines themselves, as the name implies, but also about people. The vast majority of online traffic is still driven by search engines with organic search results (people finding results when searching without advertising) still controlling a strong marketshare of digital real eastate. SEO can involve multiple different types of methods: technical methods to assist with crawlers; keyword research and implementation and content curation to name a few.

Fulfilling User Intent

As mentioned above SEO does not only involve the search engines themselves. At the end of the day a website is a product trying to attract viewers (customers) therefore one of the most impactful methods to increase the number of people who will click on your site is to 'fulfill user intent'. Generally there are three types of user intent: Informational or searching for information; Navigational or searching for a specific website and Transactional or searching to buy something. Depending on the kind of intents users will have for specific keywords or types of websites alters how you should structure your webpage and SEO. For example, if you are trying to improve the SEO for a page that sells shoes this would have a transactional intent type, from this you would know to focus of things such as SERP's and photos.

Knowing your website goals

Each website has a goal, for example a shoe store is trying to increase sales, these goals are known as Key Performance Indicators (KPI's) and can be used to measure the return on SEO investments. Some examples of KPI's are: Sales, Downloads, Email Signups, Contact Form Submissions, Phone Calls. These tend to be easy to measure in absolute terms but it is not always easy to determine how much impact SEO has had on these numbers so an aggregate of data over time is usually necessary.

How Search Engines Work

Search engines work through three main processes: Crawling, Indexing and Ranking. These three processes allow search engines to find webpages, store and organise them and finally decide which webpages best answer a search query.

Crawling

Crawling refers to how search engines find webpages. They use a huge set of computers (known as Googlebot for Google) that follow links from known sites unto new sites. If you already have a website it can be good to see how many pages are already in the index through the Google Search Console. If you wish for Googlebot to not find certain pages you can provide instructions through Robots.txt files, you may want to do this to avoid allowing webpages with delicate information to be crawled or, if you are a much larger website, to more tactically use the 'crawl budget'.

You also want to avoid having duplicate URLs through parameters. Particularly common with e-commerce sites you may have many parameters (think filters on Amazon for example) that create 100's if not 1000's of duplicate URLs. Some of this is avoided automatically but you may wish to use the URL parameters feature in the Google Search Console also.

To optimize your site for crawlers you should make sure all pages are accessible through direct links, if you need to click a drop down box to access a link it will be difficult for crawlers to follow them. You should also make sure that the mobile and desktop navigations are the same, use HTML navigations only, make sure text is not hidden and more. You should also make use of good information architecture and site maps that will help index your content appropriately.

The final thing to worry about when it comes to crawling are crawler errors. These are typically caused either when crawlers cannot access your content through client errors (4xx codes) or server errors (5xx codes). The former are often caused due to URL typos or deleted pages whereas the latter tend to be due to the URL timing out. You should avoid these errors as much as possible.

Indexing

Indexing describes the process of the crawled websites being added to an index (named Caffeine) which is a massive database of new URL's. Web pages are removed from the index when: there's a crawler error, the URL has a noindex meta tag added, the URL has been manually penalized from breaking guidelines or the URL has been blocked from the crawlers due to requiring a password or other security. If you are unsure there are URL inspection tools that you can use.

When it comes to indexing the primary thing you can do to contribute is the use of Robots meta directives/tags. These are instructions that tell crawlers how you want your webpage to be indexed. Some common tags are index/noindex which tells the crawler whether a webpage should be indexed or not; *follow/nofollow which tells search engines whether links should be followed or not and noarchive which restricts search engines from saving a cached copy of a page.

Ranking

Once a query has been searched all indexed webpages will have a rank for said search if they are deemed relevant. This rank denotes where on the results the search will end up and in general terms results that provide the best user experience and fulfillment will be prioritized. Search engines want to provide useful answers to searcher's questions in the most helpful formats.

To increase the ranking of your webpage you can make use of backlinks that work similar to real life word of mouth. Links to your websites from good and respected sources will be valued. It is also important to make sure your content is formatted well and makes use of keywords effectively. If your business has strong locality, in other words it is a local business that people go to, you also want that to be explicitly states and known through keywords in your content. As Google and other search engines have made use of machine learning components for ranking such as RankBrain the focus of user experience has only been exemplified.

In its simplest terms Search Engine Optimization is the focus on making a website fulfill its purpose more readily and visibly.