Saturday, June 20th, 2009

The Creepy Crawlies of the Web – Spiders

by Alex Gerioius

We’re not referring to the eight legged spiders hanging around outside your house, but rather we mean the spiders that search engines uses to find and catalogue content on the World Wide Web. All the search engines on the Internet are programmed to continuously crawl the web and find new web pages, tuck them into a database and figure out page rankings. This isn’t something you need to have an in-depth knowledge of or even understand past the fact that they exist, search for content, find it, use it and it gets ranked.

All you will likely be interested in is the fact that your website did get ranked and how well it was ranked. A spider is really just a robot that gets sent out on its rounds to collect Net website information as it is posted. Spiders are only able to read specific kinds of information.

If you want a real disaster on your hands, have Javascript or image text on your website and stand back and watch as the spiders are unable to read it. In fact, that type of coding can’t be read by spiders. If you want to know what a spider sees, then you have to be able to adjust your website once you know.

What, how on earth do I find out how a spider sees my site? Don’t panic, it’s very easy with a search engine spider simulator. So say for instance you have some content that is being ignored, get around it by adding image descriptions using the “alt” tag.

By the way you may also use the handy alt tag for links too. Don’t over do optimizing your website or it will just look terrible and read very poorly. The spider simulation software also tells you if your links are going to the right place.

Be careful about using any link exchanges as you may not get what you bargained for – meaning a functional link. If you don’t see a link being shown by the simulator, you will know that isn’t right.

It isn’t true that you need a whole lot of link backs to your site in order to be able to rank well on the search engine listings. If you over do it on the links you will get errors returned and not be able to access anything. Too many links with errors will mean a spider can’t view the website. The spider simulator only shows links that work, so if you don’t see a link, you know it’s broken.

About the Author:

Leave a Reply