The whole business of working to get websites to these top positions is called Search Engine Optimization (or SEO), and the industry is filled with many firms that offer these services. Unfortunately, most of these firms prey on naive customers, guaranteeing 1st page results, and delivering very little. Their business model is based on aggressive sales and high turn over. It is because most people don’t really understand how the search engines work that they end up buying services and believing the promises from these rouge companies. The goal of this article is to shed some light on the mystery of Search Engine Optimization.

First off, we are talking only about the non-paid search results on the search results page, and not the paid listings (known more commonly as the PPC, CPC, Adwords or Paid Search).

1. Search Engine Indexing: Finding & Reading

The first thing a search engine needs to do is find your website and then it needs to read your site. There are two ways search engines can find a website: Links to the site and notification of the site.

If you have other sites that link to your site, then odds are that most search engines will find your website by following the links from the other sites. If you have a new website, you probably don’t have any links going to your site and you will want to notify the search engines that your site exists. All search engines have a simple process of submitting your website address, usually found on the bottom of their website through a link called: Add Your Site.

Once the search engines know about your site, they use automated programs to “read” your website and collect information it finds. The programs are commonly called “spiders” or “bot”. They are constantly roaming through the internet visiting every website they can find and gathering information about the website. It is typical that these bots will visit a website several times a week. For some sites, they visit many times a day!

The bots are designed to go through an entire website and collect all the words it finds. It stores those words in giant databases.

So, the very first thing that needs to be done to improve rankings on the search engines is to make sure the search engines know about the website, and then make sure their spiders can easily “read” the words on the site. One of the challenges in optimizing websites for search engines is creating a site that is easy for spiders to “read” while keeping the site visually attractive to human viewers. Websites often use pictures to communicate to viewers, but these spiders cannot “see” the images or know what they are about, so a good balance of images and words is essential.

2. Search Engine Indexing: Indexing

What the spider “reads” is just as important as making it easy to find and read the site. Assuming you had a website about custom furniture and on that site, you had images of all the custom furniture you sell and a description of each piece.

The Classic

In this example, it is clear to a human viewer that this is a wood chest, named “The Classic”.  To a spider, all it would know is the words “the” and “classic”. To the search engine, it would have no idea that this website had custom furniture or offered wood chests for sale.

So, it is important that the words on the page accurately reflect the content, business or topic so that the search engines can understand what the website is about.

Once the spider visits a site and collects all the information it can “read”, it will store those words in its database, indexing and classifying based on the words it sees. If it finds the phrase “custom wood furniture” several times on a page, it will save that information in its database, and because that particular phrase was used more than once, it will believe that the page is particularly relevant to this exact phrase.

3. Search Engine Results

So now that the search engine has visited every website on the internet, and has collected every word it has found and stored it neatly in its databases, it is ready for someone to perform a search:

The search engine will quickly look into its database and find all the websites it found with the words “custom wood furniture”. It will even find all the websites that have the words “custom” and “wood” and “furniture” in any order.

4. Ranking

The critical part! The search engine now needs to make a very important decision: what order to display all the websites it has found with the words “custom wood furniture”. The goal of the search engine is to deliver the most relevant sites to the user. In other words, they want to give the user what they were expecting when they typed “custom wood furniture” into the search box.

How the search engines decide what website to show first and what makes a site more relevant than another is a tightly guarded secret. That is why no one can every guarantee that they can get a site to #1 position (or any position for that matter) because only the search engine controls that, and they will not share the secret formula. However, through trial and error, and some common sense approaches, optimization specialists know how to improve the chances of getting ranked high on search engines. Some specialists are especially good at getting sites ranked high.

The most important factors that the search engine use to determine the relevancy of a site to the search term are:

• Domain Name: If a domain name contains the search term, it will generally rank much better than one that does not.
• URL: If the URL of the page contains the search term, it will help in improving the ranking.
• Title of the Page
• Content: The if the search term is present in the content of the page, optimally  about 4% of the total words, it will help lift the rankings.
• In-bound links: More on that in the next section

5. Links

Links to your website are like votes for your website, and the more votes you get, the higher you will rank. But not all votes are considered equal to the search engines. A link from a very popular website, that has a lot of its own votes, counts as much more than from a site that has no votes for it. Likewise, links from government sites (.gov) or educational sites (.edu) count more than links from commercial (.com), organizational (.org), or other top level domain (.net, .biz, .info, .us, etc.).

How the in bound link is structured is also very important. If the visual part of the link: Custom Wood Furniture contains the search term, it is much more valuable than a generic link back to your site, like this: www.MySite.com

It is a misconception to think that links on your website going out to other websites help in anyway. They don’t. In fact, they actually decrease the overall ranking because your passing out some of your clout to the websites you link to. Outbound links should be used very sparingly, never for search engine optimization, and only to increase the user experience of your actual users.

This outline is a very high level explanation of how search engines decided what sites to show for a particular search and how Search Engine Optimization works. In practice, there are many advanced techniques to improve rankings. Some companies pay firms many thousands of dollars each month to improve their rankings. But, by applying these simple principals, anyone can improve the rankings of their site.

Marketing a website is generally divided into two categories called Paid Search (aka PPC, CPC, Pay Per Click) and Organic (aka SEO, Search Engine Optimization, Natural Listings).

Paid Search

Paid Search is basically buying positions on the search engines for specific search terms, like “medical malpractice”. You set a price your willing to pay each time someone clicks on your ad and comes to your site. Your position (top, 2nd, 10th, etc) is based on the amount you bid relative to other bidders for the same term. To get a Paid Search marketing campaign to pay off, you need to manage these keyword bids very carefully so that you don’t over-bid and waste money, but also so that you don’t underbid and simply  get nothing in return. The entire system is pretty (no, very) complex typically requiring daily, hands-on management.

The quality of your website has a direct impact on how well a Paid Search campaign can do. A well designed site will have a much higher rate of “return” (more people will call you after coming to the site), so that the actual costs leads you get from the clicks will be reduced. In essence, you will save money because your advertising is much more efficient. That goes back to my argument that a well designed site will save money in marketing because it does a better job turning marketing dollars into real clients.  Using poorly designed sites on Paid Search campaigns will waste a lot of those advertising dollars.

The amount you decide to spend on Paid Search is very controllable and measurable which can be changed at a months notice.

Oragnic Search

Organic is a completely different approach. Every search engine will show two categories of results: The Paid sponsors and the regular listings. The goal of Organic is to “convince” the search engines to list your company, for a given search term, at or near the top of these free listings. They are free – you don’t pay the search engines for them – and they bring in a lot of traffic. However, there is no direct method of getting your site to those coveted top positions. That is where SEO/Organic marketing comes in.

The way search engines work is that they send out these programs called “spiders” that visit every website in the world, and revisit them, on a continual bases. These programs collect the information they find on the websites and store that information in a database so that when a user does a search on a term, like “medical malpractice”, the search engine can then just query its database and display all the websites that contain information about those words.  The complexity comes into play on the “ranking” or the order in which the search engine decides to display these websites. Their goal is to present the most relevant, or “on topic” sites on the top. They keep their algorithms under tight wraps so no knows exactly how to get their website on top.

However, through experience and trial and error, SEO experts have figured out how to design websites, position the sites and manipulate the organic search results to get their listings ranked high. This processes is called Search Engine Optimization and is an on going process of manipulating things under our control to get higher rankings (which are not in our direct control).