fplanque: IT world

When looking at the sitemaps protocol which is now endorsed by Google, Yahoo and MSN, I can't help but crying about how obscure the documentation is, especially for the <priority> element.

Please note that the priority you assign to a page is not likely to influence the position of your URLs in a search engine's result pages.

Okay, so what's the point?

Granted that setting all priorities to 1.0 will not make the urls rank higher than urls from other sites. But we're talking about position of URLs here, not sites.

Sometimes, the same site appears multiple times in search results, with different pages/urls. In that case, if priority doesn't influence which URL comes first, compared to which other comes second, then what's the use?

For example, on a blog, the same info can be found on a post's permanent url, on the homepage, on the category page, on the archives pages, the RSS feed, etc.

Sometimes the search will return several of these locations. If the priority can't be used to tell that the permanent url would be the best choice to put first, then... I don't get it!

Does it mean that priorities are only used to determine what gets crawled first? If it does, then it means that maybe the 100 top priorities will be indexed and the others won't! So the top 100 may appear in search results and the other may not!

Present vs. not present! That's what they call "not influencing the position'?

Again, if it doesn't do that, then what does it do?

All I can think of at that point is the priority being an alternative to <changefreq> : a site gets a certain number of reindexes a day, and high priorities pages will be refreshed more often that low priority pages.

That would comply with the definition of that <priority> does NOT do...

But then... it doesn't make sense with what it is *supposed* to do:

it only lets the search engines know which pages you deem most important for the crawlers.

Or by "most important", are we supposed to understand "most frequently updated"?

I really wonder who it helps to have that spec being so obscure...

Google.be 28-sept-06

Everybody’s been saying lots of things about the Google.be case, especially that the Belgian newspapers should have used robots.txt to tell Google what not to index. And that the fact they did not use robots.txt clearly show all they were interested is in getting money from Google…

Well, friends, I’m no lawyer or legal expert of any kind, but I’m French… and that lets me read and “almost” understand the terms of the ruling… I guess…

I think the ruling makes it pretty clear what the Belgian newspapers want, and I think this has been mistunderstood:

• The papers welcome Google to index and display their news as part of Google News! (or at least they don’t care)
• The papers’ particular online business model is that news are free, but access to archives require payments. Example here.
• Once an article falls out of the news category and into the archives category, it should not be freely accessible any more.
• Google, via its world (in)famous Google Cache, often makes the content available forever, or at least for a very long time after is has gone off the official site’s free area.

I guess that’s it: what the Beligian paper really want is a way to get the content out of Google News once it is no news any more.

Now, I’m no robots.txt or Googlebot expert either, but from what I understand there was no convenient way for the papers to tell Google that it is okay to index some content for, let’s say 2 months, but not keep it in cache after that delay.

Goggle made some general comments on the case on their blog, but:

• They are not allowed to comment specifically on the ruling, so it’s not that useful;
• They failed to show up at the trial, which is quite unbelievable… but would make it almost believable they fail to understand the real issue that has been raised…

Note: again, I’m no legal expert. Just trying to make a little sense of all this noise…

Be social: digg this!