Sitemaps are back and the best that they’ve ever been. They’re a far cry from putting up some half-assed page with maybe 25% of all your site’s URLs. Those were pretty useless and really just for the search engine spiders anyways, so why not take the page out of the equation? The Sitemaps protocol does just that and being that it is supported by Google, Yahoo!, and Microsoft/Live, it has the teeth that a standardization needs to catch on.
I’ve been using the sitemap XML format for some time now with my sites and it really does work. The spiders find everything on your site. The only catch is that you need to submit your sitemap to the search engines for it to get read. Yeah, I know, you should just be able to use the SITEMAP: http://www.the URL of your sitemap here.xml bit in your robots.txt file, but the search spiders really don’t seem to pick it up like they do when you manually submit it. Being that this is the case, management tools have been build for dealing with your sitemap feeds. Google was the first. Yahoo! soon followed. Microsoft, as is usually the case was markedly behind the curve, but has released their system to the public just four days ago.
What follows is a review of the three systems. Generally, they all do the same thing, but much unlike the standardization that when in to the sitemaps protocol, their system are rather different from one another…
Google Webmaster Tools
Far and away my favorite. They have been tinkering with it for a great long time and have gotten it to a place, where I feel that it’s a solid application.
The thing that I like the best about this system is that it just seems to get better. It’s like there are a team of guys sitting around who are actually using it on sites and when they think of something that could make it better, they do it. But beyond this novelty, I’d have to say that my favorite part is the Statistics tool. There are tools in there above and beyond other sections, where you don’t need to use this system to pull off those functions; just phrase a query right. With Statistics, you can find out when you were last crawled and the overall PageRank of your site.
One tool that I find to be something of a downfall is Diagnostics – Web Crawl tool. At first it seems very informative, but then you realize the the 404 errors it finds on your site were really just flukes on the server and the pages are still there.
For a little while, things were getting kinda clunky with this system, but now, everything seems to be heading in the right directions. They even have a mobile section developing in there, which is pretty cool.
Yahoo! Site Explorer
The middle-comer to the game and a good system. Unlike Google and Microsoft, I think that they took longer to work on the system behind the scenes (or just bought out a smaller company.) So, when it came out, it was working very slickly from the start.
First of all though, I apologize for the URL. The site isn’t the easiest thing to find, at least for me. I think going here, clicking on “Site Explorer Home” and then logging in will get you to the management page. This was of course my first initial annoyance with the system overall–trying to find it. Once in though, I’ve found it to be a nicely setup page with different levels to manage different feeds for your sites. But, that’s about it. There are not any additional functions in here. In some way, I appreciate that. No goofing around. You want to submit a sitemap? Then this is where you do it.
They do have some gimmicky items you can put in there under the Badges section if you want. They also have a blog if you want to keep up on the latest and greatest. Although, it’s ironic that the one really big stumbling block that I see for their system is that their updates aren’t so late and great. I have yet to see a notification that my sitemap has been crawled before I tell it to do so and I’ve waited up to a month with some sites. So, that’s not so cool and requires a great deal of handwork, whereas the Google system swoops across your site very often.
Microsoft Webmaster Tools
Still, very, very much in beta. The basic system is functioning, but like most things Microsoft, it was released a bit too soon. This was of course after dropping the ball for some time on getting a system like this going. As you can see in the comments, people have repeatedly asked, “Um, why are you telling us to do this, when you don’t have a system to work with it?”
Since MS really is far out of the #1 and #2 spots, it would be expected that they would bring out a set of tools that aren’t all that amazing. But, oddly enough, they’re actually pretty decent currently. I happen to like the Website Status section under Summary. It’s the closest that I’ve seen MS get to creating a PageRank score. But, it does show how pathetic MS has been in indexing. For this site, they only having 53 pages, which falls far short of the 3,200 or so that actually exist.
And then they have some keywords scorings in there are well. This tool set is not nearly as robust as Google’s, but if they keep working on it, it might get somewhere yet.
There are a couple of serious bugs with the system. For one, it appears to be much happier in Internet Explorer, which is no surprise. Another is that the FAQ links to nothing. Another is that their “click to expand” function to list all your sites keeps tossing the one you just clicked on to the bottom. I find this annoying and would prefer a static list that was alphabetical. Ah, and then some of the sections just vanish. For instance, while writing this, all the Summary for a site was gone. An hour later, it was back. Yes, beta. Very beta.
I don’t know how their spider regularity is at this point, since this service is very new. As of using it for three days, they still don’t appear to have picked up any new pages. Then again, this could be a reporting bug. We’ll have to see how this plays out over time.
Which one do I love the best? Google, of course. But the truth of the matter is, you’re going to have to use all three, so get used to it! It would just be nice if you could use the account on all three systems to manage your account. Microsoft is actually better than Yahoo! and Google on this because you only need to have a one of their IDs which can be set up with any email.
In the end, I think we should all be thankful there aren’t XML hacks like what you have to do to get a site working in Explorer 6…