We have all been there, “how the heck do they always get #1?” It is a constant frustration for many a client and, well, even myself occasionally. The fact is that much of the time there are a few solid reasons behind the search engine success of any website and it is important to learn what these reasons are before trying to compete. How is this done? Therein lies the subject of this article; how do you determine what your competitor has done to win the search engine war?
Demystifying your competitor’s success requires you to put on your
detective garb because you are going to have to investigate all aspects
of their website; even the deepest darkest corners. In the following instructional
I will lead you through a hypothetical investigation of a competitor who
is ranking for the phrase “voip services”. In each step I
will choose the more popular result that I find when I do similar competitor
analyses professionally. So please take note, the sample is only the most
popular result; occasionally there are truly baffling cases of competitor
success which have required heftier investigation leading to differing
conclusions of what they did to succeed.
STEP 1: Start with the Basics: Is The Site Optimized?
Visit your competitor’s website and look at the content on the
page. Does the keyphrase they are ranking under appear often throughout
the body text? Does the phrase appear in the headings, Meta Tags, Title
and in their menu? If this is so it generally means they have focused
completely on attaining the keyphrase they are ranking highly for. In
other words it is quite possible, and even likely, that they have sacrificed
other keyword goals in order to achieve this single goal.
Example Results of Step 1:
“ The competitor’s page has utilized the ‘voip services’
often but not in a spammy manner (not obvious overuse) throughout the
body text on their home page. The page topic is clearly focused on this
phrase because the Title tag states ‘VOIP Services – by VOIP
Business Systems’ and the headings and menu items are clearly related.
Conclusion, I have no doubt that this site has focused on this phrase
and that a search engine spider will have no problem logically identifying
the subject of this page as ‘voip services’.”
STEP 2: Is Site Structure Playing A Role In Their Success?
Site structure is a fundamental reason for search engine success. I
cannot stress enough just how important it is to have a website that can
be freely and simply indexed by search engines. The fact is that the more
legitimate and useful content that a search engine has access to on a
website the better the environment for top rankings. To put this inquiry
simply: “how well is the website indexed?” Each search engine
may react slightly different to the technology used within a website so
let’s tackle the 2 largest and most revealing engines:
Test Google and Yahoo: On each search engine use the following syntax
replacing “competitor” with your competitor’s site:
site:www.competitor.com . This search will reveal how many pages that
each major engine has indexed within the competitor’s site. Now
that you have a list of the pages indexed, look and see if each indexed
page has its own title and description. The answer will determine whether
each page within their site has been individually optimized.
It is also crucial to look at the URL’s for each link. Does each
link appear to have many symbols such as “%&”? Often a
website with a lot of extraneous code in their URL’s will not be
100% indexed because the search engines tend to shy away from pages that
are created dynamically (by databases – which these characters state).
If there are no extraneous characters it may be quite revealing if you
know the site is far too big to be anything but driven by a database.
In this situation it is apparent that the competitor’s website has
been tweaked by a programmer to allow the search engines to index everything.
This tweak is extremely common and highly recommended for anyone who has
a database driven website with unfriendly URLs (contact
me for more info).
Another useful and revealing tactic is to determine whether your competitor
is using Google Sitemap to improve the indexing of its website. To find
this out with reasonable chances of success, try entering the following
URL into your browser’s address bar: www.competitor.com/sitemap.xml.
This is the default file name for Google sitemaps so if your competitor
has one this will often provide you with a resulting page full of xml
is an example of what you would see if your search were successful
(from StepForth’s website). If you do find a sitemap then it is
reasonable to assume that your competitor is relatively well informed
on the latest search engine tactics and that Google Sitemaps may play
a small role in their online success.
Comparing Actual Site Size
Finally you need to know exactly how many pages your competitor actually
has online. This way you can determine how well their website has been
indexed. To do this there are a variety of software options but one simple
and free option is to use Xenu’s
Link Sleuth. Xenu is a free site analysis tool available here;
in the past I have found it very simple, small to install and unobtrusive.
Install the program and select a new project where you just need to enter
the address to spider and let it go to work. Ultimately you can ignore
a lot of the information because you are not trying fix dead links. The
key is the “Statistics for Managers” section located at the
bottom of the report. The number of pages within the site can be found
next to the “text/html” statistic.
Example Results of Step 2
“ I saw that Google had indexed about 9000 pages and Yahoo had
indexed 138,000 pages. I am quite impressed with these indexing; obviously
my competitor’s site is easily indexed by search engine spiders.
To back this up I see that the URL’s do not have any extraneous
characters yet they are also very long… this database-driven website
must have been tweaked for search engines.
"Yahoo had more results than Google but that isn’t a surprise
since Yahoo tends to be more forthcoming with its site-specific queries.
A Google Sitemap does exist on the competitor’s site but because
Yahoo has also indexed the site well I don’t suspect it has played
a significant role. As for the ratio of actual pages to those indexed…
it appears that Yahoo has actually indexed more pages than my software
found. Overall I would say that this competitor has spent a lot of time
making their site search engine friendly. I should compare these results
with my own site… perhaps I need to follow suit!”
STEP 3: Where, Oh Where Do They Get Links?
These days another fundamental issue are backlinks (links pointing to
their site); just how many have your competitors got that you don’t
have OR just how good are they? For example you may find that your competitor
has thousands of links but when you research them they appear to be less
than relevant to their site topic. Only relevant links offer any advantage.
In other words, if your site is about VOIP and you have a link from a
hair salon you are not likely to get an iota of benefit from this link.
That said, if you have a link from a telephone sales website then there
is an obvious correlation in relevance and your perceived search engine
popularity will increase.
In order to discover this information you have two options. One, perform
a manual search of your competitor’s backlinks where you can randomly
review the quality of their links (time consuming but ok for small sites).
The other option is to purchase a program like OptiLink
which is designed specifically to simplify the investigation process;
especially for large competitor sites. For the purposes of this article
I will stick to the manual option but I do recommend checking out OptiLink
because it also provides other amazing analysis tools that will help you
get better rankings.
Open two web browsers and then follow these instructions:
In one browser visit Google and type in the following: link:www.competitor.com
The resulting list will include a sampling of the backlinks that Google
‘sees’. Now I note “sampling” because Google is
not very forthright with this information. For example I can state with
little doubt that Google actually sees just as many backlinks that Yahoo
does yet Google always shows fewer in their results.
In the other browser visit Yahoo and type in the following: linkdomain:www.competitor.com
The resulting list is likely a very realistic assessment of all the
notable links your competitor has online.
Now that you have a list for each major search engine you can tackle
randomly investigating the links within each.
Some Points to Look Out For:
Are the sites relevant to your competitor’s site?
If they are not relevant you can consider them a low quality link with
likely zero benefit.
Do the sites appear to be reputable? In other words
do they have obvious Spam? If there is obvious Spam you can ignore this
Do the sites have decent PageRank? (use the Google Toolbar
to deduce this)
Low PR sites (>4) are generally less helpful. High PageRank sites (5>)
often have better impact because they are considered reputable so most
outgoing links will be votes of confidence for the destination site. Please
note that this is a topic thick with opinions but these are fair general
Do they appear to be link farms? If they are, this link
can generally be discounted.
Are the links paid? If you notice that the links shown
are paid links take note of the site and consider contacting them for
their rates. Even if you decide not to proceed with their service it can
be a revealing piece of information.
Example Results for Step 3:
“ I decided to review 50 different back links on each search engine
and I discovered something very interesting. First of all, of the 100
links that I reviewed I noticed that my competitor actually owns the sites!
I figured this out by reviewing the Whois of each site because they appeared
to be so promotional. Each site is informational, well designed and obviously
created to build traffic to his main site. This is not a tactic that I
want to follow due to the time it takes and from an ethical standpoint
so I chose to focus on the other links I found. Unfortunately it appears
that the other links were not too revealing; none of them appeared to
be of especially high quality and most appeared to be from an old link
building campaign because they were link farms (an old and dangerous tactic).
"I did have one lucky hit! I found about 10 websites where my competitor
is paying for links. These sites appear to be of very good quality and
the number of outgoing links is very low. I am going to find out if advertising
on them is affordable.”
STEP 4: Your Site – Compare and Contrast
Now run all of the same analysis you just did for your competitor on
your own site and record your discoveries.
STEP 4: The Resulting Conclusions
Compare and contrast the information that you have compiled from your
competitor and your own website and create a game plan.
Example Conclusions: The Game Plan
In point form this is what I believe I need to review on my own site in
order to compete:
1. I must simplify the URL’s within my website. My research has
shown that my site has several roadblocks because my site is only 20%
indexed in both Google and Yahoo.
2. I need to optimize each major page within my website paying particular
attention to the customization of each title tag. I noticed that each
of my competitor’s top performing pages (despite being deep in their
site) was well optimized.
3. It may not be the biggest help but I am going to add a Google Sitemap
to my website once it has been optimized.
4. I need to attain more backlinks. I see how my competitor has gained
backlinks but I have no interest in pursuing this angle. Instead I am
going to hire a link building firm as well as start my own blog on VOIP
emerging technology that I hope will produce enough interest to get some
Other Forms of Competitor Analysis
The analysis that I outlined above is used when you are trying to beat
out a competitor under a particular keyphrase target. But this is only
one form of competitor analysis.
For example, perhaps you know a competitor down the street from your
office is doing exceptionally well online. In order to find out how they
were doing well you would approach an analysis differently. You would
first need to discover what keyphrases the competitor is targeting and
what exactly they are doing to obtain rankings under them. Additionally
you would need to discover if this competitor is profitable for reasons
other than search engine rankings. For example, they may be acquiring
traffic from another website through a paid advertisement or a mutual
partnership. There are many ways that a business can profit online and
once in a while I find a completely new and occasionally cool concept
when I research competitors for my clientele.
Just a reminder, the steps noted above represent just the tip of an iceberg
for researching your competitor’s success. If you are finding that
significant pieces of the puzzle still remain unanswered after following
these recommendations please do not hesitate to give me a call (250-385-1190).
About The Author:
Ross Dunn is the CEO of StepForth
Placement Inc., a search engine marketing company founded in
1997 and based in Victoria, British Columbia, Canada. StepForth provides
cutting-edge search engine optimization services that provide highly successful,
targeted results for its clientele. Ross Dunn is a Certified Internet
Marketing and Business Strategist (CIMBS) with a background in web design
and online marketing. His broad Internet experience in combination with
a talented staff has made StepForth a name synonymous with top results.