Last updated on 13th June 2023 at 21:22 by Alex Nicholas

Search engine optimisation is the practice of marrying a website to a search engine, whilst giving yourself the best chance possible of being found by your target audience.

SEO is all about two things, the relationship your website has with search engines and focusing on playing the long game. How it works and your outlook on it must always start with giving the user what they want whilst allowing the website owner to easily show off their content as well as making it crystal clear to the search engines what each individual page and the site as a whole is all about.

Without this at the forefront of practically every SEO decision you make, your website is destined to struggle. It's only when you follow the basics set out below that you can truly optimise your site for both users and search engines.

How SEO Works – Crawling & Indexing

Crawling and indexing websites is optimal by ensuring the following:

Crawling

  • Making sure all pages that we WANT in the search engines index don't have the robots “nofollow” meta directive in them. These pages MUST have the “follow” directive
  • Navigational links DON'T use the “rel='nofollow'” attribute.
  • Your robots.txt file isn't blocking pages you want to be indexed by the search engines
  • Your server is able to deliver your content quickly to the user and the search engine crawlers. If Google sees that your server times out enough (5XX errors) it will algorithmically demote you.
  • You block unnecessary crawlers, for example, certain SEO monitoring tools, from using valuable server resources
  • Your website is well-designed and easily readable. Fancy designs all too frequently have an adverse effect on a site's ability to rank highly in the SERPS. Never make visitors search for menus or pages on your site
  • There are no broken/dead links on your site
  • Poorly handled canonicalisation of duplicate content

I feel that I should also comment on the often poorly understood crawl management. Crawl management is not crawl budget. Crawl budget is a term used by many within the SEO industry, but in my opinion, is not accurate.

Crawl budget is set and controlled by the search engines and there is nothing a Webmaster can do to change the amount of “budget” a website receives. All we can do is control how a website is crawled so when you read another article on taking care of your “crawl budget”, remember that it's not the same as managing crawl.

If you want Google to crawl your pages more, then build more good links to your pages.

Indexing

  • Make sure search engines are able to fetch, render and index all the content you want them to
  • Duplicate content must be eliminated. Although there's no direct demotion for duplicate content, by having it on your website you're forcing the search engines to choose between URLs, which can and does confuse the situation
  • Use only search engine supported Schema markup, which across the board is JSON-LD
  • Make sure that your HTML code is being used in the correct way
  • Make sure your website is coded “cleanly” and any excess bloat and/or redundant plugins are removed
  • Use “ALT=” text to describe all images

How SEO Works – Unique Content

Defining good quality content is problematic for many SEOs out there, however, what should never be disagreed upon is the fact that we should be producing unique content (that isn't simply a rehash of the same old blogs, etc) that must add value to the reader and must be relevant to the search query.

Combining these two SEO elements is crucial when considering search engines have become very good at spotting very similar content that's designed with one purpose in mind, to rank highly in the SERPs. Content that satisfies the users' needs and expectations is what I consider good content.

However, one thing to bear in mind is that it can be tempting to go all out and cover a topic in magnificent and unadulterated detail, this is often a mistake. Users expect to have their questions answered so resist the urge to fill every page with 5000 words, especially when the top results are up there with roughly 2500 words. Answer the query completely, but no more.

Competing in a competitive space

In my opinion, and especially when you're competing in a competitive niche/vertical with established companies, you should pay close attention to the style, design, content, and format that your competitors are using and putting out. I've long held the belief that search engines train their algorithms to expect certain things in certain verticals, so if you're stepping into a competitive space pay close attention to the content type and design of the top-ranking sites.

How SEO Works – Well-Organised Information Architecture

User experience (UX) is, or at least it should be, near the very top of your priority list when it comes to designing your website in the mobile 1st era. With good UX you can ensure that your visitors can not only find what they want quickly and easily but also consume your information effectively, or carry out the task they came to you for in the first place without having to search around and break out into a tut (come on admit it we've all done this). This in turn creates a positive experience for both the user and the website crawlers.

It's a myth to believe that if a user visits a page on your site and then leaves within a short space of time that this is a bad signal. On the contrary, if someone is looking for an answer and yours is relevant and set out in an easy-to-digest format, I'd expect them to be able to get the information they're looking for and then leave. This to me satisfies user intent which is a good thing.

However, you may want to show articles, posts, or pages to a visitor that they might also find helpful or related. By showing these pages are on your website with carefully placed internal links you can gently guide users through your site and increase their awareness of your products or services, thus increasing the chance they'll see you as a valuable resource.

A caution on ads

Good UX will also dictate how you show ads if indeed this is something you intend to do. However, I strongly advise against aggressive advertising and interstitials that dominate screen space as this is one of the quickest ways to get you wrestling with the Panda algorithm which will demote you and drop your website's organic visibility.

How SEO Works – Matching The Right Content to the Right People

At the very heart of any search engine algorithm is one outright necessity, the search query must return accurate results that reflect exactly what the searcher is looking for, if it doesn't then it's failed in it's objective.

The positive for any Webmaster or SEO is that we can make the algorithm's job much easier by focusing on semantic SEO making the topics of our Web pages crystal clear by adding important keywords and phrases in all of the places they need to be, and just as importantly by leaving out keywords and phrases that might dilute or confuse the algorithms and should be on another page.

I would go so far as to say that 80% of getting a website to rank highly is how each Web page implements the following 4 things:

  1. Keyword research: Needs to identify queries that the website should target to drive growth. These keywords must suit/match the persona of your ideal customer so that the traffic driven to your site has the potential to provide business for you. Proper keyword research will find gaps or missed opportunities that grow your bottom line. What you must never do is chase countless keywords on numerous pages in an aggressive content strategy that chases worthless visitors and all too often leads to keyword cannibalisation and a weak website. Focus is key.
  2. Detailed content strategy: Identifying the right keywords is great, but they're nothing without a detailed content strategy that will bring visitors to your site and into paying customers. By having an SEO content strategy in place you know exactly where you need to position yourself and how you're going to get there.
  3. Adjusting keyword targets: Arguably the best opportunities lie within the underperforming content you already have on your site as these pages will already have some standing in the SERPS. The focus here must be in adjusting the target keywords to better fit the vertical the page is in, with the results often coming much faster than creating new pages. This invariably leads to rewriting some, if not all of the content.
  4. Matching searcher intent: You may well have already heard of “intent” within search queries, if so you should also know that it goes hand-in-hand with your keyword targets, for example:
  • Informational intent: These are searches for answers to specific questions such as “the best way to handle website crawl management” or “what's the weather like tomorrow”.
  • Navigational intent: Navigational searches are searches with a very clear intent with regards to navigating to a specific website. For example, a user might enter “digital cornerstone” into the search bar to find this website, rather than entering the URL into the search bar or heading here from a bookmark.
  • Transactional intent: People searching with transactional intent are buyers with a credit card in hand, ready to purchase whatever it is they're looking for. They're generally on the hunt for the best price for a product and will hop from site to site if they're not already aware of which website has exactly what they want.

Okay, I'm not going to lie to you, I've left this section of this article to the end for one reason, in many instances they don't matter anywhere nearly as much as many search marketers will have you think. Google uses backlinks and often it's anchor text to find and validate a website. Links cut through the noise created by the algorithms that deal with content etc.

Links from one website to another give visitors the opportunity to gain more information from content on a topic that's related to the content they're already viewing. Note that the best links should be placed in content that gives meaning to the link itself, the overall context of the website doesn't matter that much.

The vast majority of websites don't attract many backlinks due to the nature and quality of their content. So if you set about buying links and participating in shady tactics that will unnaturally inflate your link profile you run the risk of suffering an algorithmic penalty, or worse still a manual penalty, meaning a search engineer has manually viewed your link profile and deemed that you've manipulated it.

Search engines know that most websites don't have strong backlink profiles and will use other algorithm signals to rank them in their SERPs. This is why your content strategy must be on point.

Summing up

Search engine optimisation is a complex and often daunting discipline that almost never comes with a quick fix, it takes time, understanding, and most of all patience. However, when done correctly a website that has the fundamentals of SEO in place will hold it's own in the SERPS and will always stand strong when the algorithms are updated.

A well-optimized website has a balanced relationship with both users and search engines.