Spider Simulator Google Gadget
I made a gadget from Aaron Wall’s search engine spider tool source code, available from his SEO tools page.
I have had my version of this spider tool on my other site for a long time, but I wanted to make a gadget that showed the results within the gadget, instead of throwing you off to another page. After some tinkering, I have figured it out…
Here is the live gadget (give it a try)…
This tool will show you very quickly some of the main things a spider sees on your webpage (or someone elses). It displays the title, description, keywords, how many words, keyword densities, and outgoing links.
To add this gadget go here or just click the “+Google” button beneath the gadget.

February 7th, 2008 at 2:53 am
Pretty cool… do most spiders ignore robots.txt ?
March 6th, 2008 at 1:14 pm
Nice question Tim, I too would love to know, Pat. Do spiders ignore robots.txt?