promotion Tip:
Where Did My Site Go?
by
Larisa Thomason,
Senior Web Analyst,
NetMechanic, Inc.
It's every webmaster's nightmare. One day your site is
steadily climbing in the search engine rankings and building solid link
popularity. But the next week, you find that it has either vanished entirely
from the search index or the rank has dropped so low that nobody will ever find
it. Here are some hints to help you find, fix, and hopefully avoid this
traumatic situation.
Check Your Server
First, try to access your site online and make sure the
server is up. This may seem really obvious, but just ask anyone who works in a
tech support center how many calls they field from confused customers whose
computers, scanners, etc. don't work. Not because the products are defective,
but because they need to be plugged in first!
Most Web hosting companies boast at least a reliable "99%
uptime," but perhaps the search engine spider cruised by during that crucial 1%
and found nothing. Quite a few human visitors may have had the same experience.
Once a page is dropped, it stays gone until the spider revisits your site.
Search Engine
Starter lets you easily
deep submit
dropped pages, but remember that it may be months before the spider crawls by
again!
Since you can't possibly sit and monitor your server 24 hours
a day, NetMechanic's
Server Check Pro tool will do it for you and alert you by pager or email if
your server isn't responding. A server problem is always the first thing to
check because correcting it is usually as easy as making a phone call to your
hosting company. That's a lot easier than redesigning your site.
Review Your Robots.txt File
The robots.txt file tells spiders which pages they're allowed
to visit and lists those that are off limits. Have you made any changes to the
file recently? If so, you may have inadvertently closed the site to one or more
search engine spiders. Or, you may have placed it in the wrong directory and the
spider couldn't find it.
Review this
excellent discussion of the robots.txt file to learn more about its
important function.
Review Your Web Host's Terms Of Service
Perhaps your site has moved into a "bad neighborhood." That's
search engine-speak for a server that hosts either adult sites or sites with a
reputation for spamming search engines. Search companies don't bother to ask why
your site is hanging out with these unsavory elements: they just ban the
server's IP address entirely.
So if you're having a problem getting indexed, check your
server logs to see if any search engine spiders have visited. The online
Search Engine
Dictionary maintains a
list of spider bot names so you can check to see who has been by. If you've
submitted and waited months without a single visit, then perhaps your site is
hosted in the wrong place.
Check your Web hosting company's terms of service (TOS)
agreement to see if they allow adult or gambling sites. Both have a reputation
for being inveterate search engine spammers (even though not all are). It could
be that the company began allowing those sites after you signed up and the
search engines have caught on and banned the IP address.
In this case, your only real recourse is to move your site to
a new hosting company. Refer to our
Seven Questions
To Ask Your Web Host article for tips on evaluating hosting companies. It is
possible to appeal your ban with a particular engine, but that process could
take months and cost you a lot of visitors.
Have You Paid Your Bills?
If you're using a pay for placement (like Overture) or pay
for indexing (like Inktomi), or pay for inclusion (like Yahoo) service, make
sure your payments are up to date. Check your email for notifications from the
company.
Many webmasters maintain a separate email account for domain
renewal, search engine and directory submission, and other related site issues.
It's easy - but dangerous - to forget to check that address regularly. Most free
email providers either close or delete the mail from inactive accounts and
accounts may be termed "inactive" if you fail to check them at least once every
30 days.
Fortunately, this is the easiest problem to fix. Login to your account, make
your payment, and your site is back in the index!
Have All The Pages Disappeared?
Well, now we're getting to the hard stuff. The first four
suggestions pointed to outside sources that could cause problems. But it's just
as likely that the problem is with your Web site content itself.
Before you change a thing though, check to see if every
single page of your site has been deleted from the index. Most likely there's a
problem with just a few pages. That points to an algorithm update at the search
engine.
The "algorithm" is the super-secret formula each search
engine uses to rank and/or penalize Web sites. Search engines change them
constantly to thwart spammers and to provide more relevant results to searchers.
Each time the algorithm changes, your page rank may change too.
You'll have to look at your page content and HTML code to
determine which design or coding technique is causing the problem. Compare the
dropped pages (or those whose rank dropped precipitously) with your high-ranking
pages. Maybe you'll see that the dropped pages lack
header tags or
ALT text on
images. Perhaps the pages have very little text content. Look the dropped pages
over carefully and try to find a common element.link,links,site,sites,search,good,online,mail
,company,custom,customer,line,sale,business,service,design,search engines
submission,search engines submissions,url search engine submission,search engine
submission,website search engine submission,search engine submission
software,search engine submission service,canada search engine
submission,canadian search engine submission,search engine submission
companies,search engine submission tips,search engine submission company,search
engine submissions,international search engine submission,search engine
submission script
Evaluate Your Site Content
Let NetMechanic's
HTML Toolbox
tool scan the problem pages before you spend hours combing through your code. It
will identify
simple coding errors that may not affect page display in a browser, but may
keep the search engine spider from indexing all of the page's content.
Once you have an error-free page, check for other issues that can also affect
search engine rank.
- Before and After: If you kept a backup copy of the site,
compare the pages before the change and after. That terrific new design
you implemented may be shutting visitors out instead of drawing them in.
- Server Redirects: Try to avoid
META
refresh tags. Some spam sites lure traffic by purchasing expired
domain names that have a decent search engine rank and
backlinks.
Then, they use a META refresh to direct the unsuspecting visitor to
their spam site. Search engines are wise to this tactic and may penalize
sites that use META refresh tags.
It's much safer to use an HTTP header 301 (permanent redirect) when you
need to change a page location. Check with your Web host to see if you
have access to HTTP header information.
- Keyword Stuffing: Yes, it's important to select the best
keywords and keyword phrases for your site and use them inside header
text, ALT tags, and page content. But it's easy to get too much of a
good thing. Search engine algorithms penalize sites that over-use
keyword phrases.
Unfortunately, it's hard to know if you've used a phrase too often or
not often enough. NetMechanic's
Page Primer
tool will calculate your page's keyword density score and alert you if
there's a problem.
- Link Farming? Remember the importance of keeping your site
out of a "bad neighborhood?" Well, it's just as critical to avoid
linking to sites in bad neighborhoods or to sites that are totally
unrelated to your site's topic.
Read more about the
dangers of link
farming in our April, 2002 Webmaster Tip.
A great way to avoid these types of problems is to use
NetMechanic's handy, affordable coding and search engine tools. Even if you've
made a simple change, always use
HTML Toolbox to
check the changed page for errors.
Then use NetMechanic's
Page Primer
tool. It will scan your pages for problems and provide engine-specific tips to
improve page rank. You'll know if you've used your keywords too much (or too
little), if there's enough content, or if you've inadvertently used some design
technique (like hidden text) that got you banned.
| 
![[Under Construction]](images/undercon.gif)
