Archived blog with a focus on DotNetNuke news, tips and tricks, DNN SEO, and insights and opinions about the DNN community at large.

First time here? You may want to check out the blog archive, subscribe to the RSS feed, sign up for free email updates, or follow me on Twitter. Thanks for visiting!

Point Google to Your Sitemap Via Robots.txt 

XML SitemapThe big four search engines (Google, Yahoo!, Microsoft Live Search and Ask.com) recently united behind a standard called sitemaps “autodiscovery.” So instead of submitting your xml sitemap to every search engine individually, you can now add the following line to robots.txt and your sitemap will be picked up automatically:

Sitemap: http://mysite.com/sitemap.xml.

This is not to say to you should forgo tools such as Google’s Webmaster Central or Yahoo’s Site Explorer, but it’s a great first step to expose your site to the major search engines quickly.

On a side note, back when DNN 4.5 was first announced I promised a review of DNN’s ability to create an xml sitemap dynamically. Well, it turns out there is not much to review after all. All DNN 4.5.x does at this point is create a sitemap according to your site’s navigation scheme as stored in the Tabs database table without Admin and Host menu items. For my site that means that all blog entries are not included, which is exactly the kind of content I want to tell Google about. Here is my “hand-massaged” sitemap created with xml-sitemaps.com.




Comments

Dan Dan says:

xml-sitemaps.com is a gem!

Thanks for the tip.

Tom Kraak Tom Kraak says:

I'm glad you found it helpful.

Dan Dan says:

Tom: the link pointed to in " All DNN 4.5.x does" is your sitemap... but this was NOT generated by DNN 4.5.1, but rather from the sitemaps-xml utility- correct?

Tom Kraak Tom Kraak says:

Sorry Dan, I knew this was going to be confusing.

Here is my sitemap generated by DNN:
http://seablick.com/sitemap.aspx

And here is my "custom" xml-sitemaps.com sitemap:
http://seablick.com/sitemap.xml

Feel free to ask more questions.

Dan Dan says:

Ah- my bad for not checking them... doh!

How about this: does it matter that the default sitemap is generated- if I am to use a different one (namely, one generated by xml-sitemap.com)?
As long as I submit/tell the 'bots which one to use, they'll ignore the other(s)?

Tom Kraak Tom Kraak says:

That's correct. No bot will "automatically scan" the root of your site for any sitemap file.

You'll have to tell the bot about the existence of your sitemap file by either submitting to the search engines directly or by pointing them to it via robots.txt.

Dan Dan says:

Great stuff!

Thanks for the enormously useful blog.

Tom Kraak Tom Kraak says:

My pleasure. Just keep telling your friends about it :)-

Also, what other topics would you like to see covered?

Dan Dan says:

I found your short blog about "keywords" to be provocative... It seems that on the DNN "page settings" , there are lots of places for keywords (page title,description,keywords), and I wonder about their utility and relative effectiveness. Any chance you could elaborate upon this?

Tom Kraak Tom Kraak says:

Sure Dan, I'll shed some light on DNN's Page Settings in terms of SEO. Look for it early next week. Thanks.

Michael P Michael P says:

Tom, any idea how to handle pointing to the correct sitemap when you have multiple sites running on the same DNN installation?

Tom Kraak Tom Kraak says:

Mike, you'll have to submit the sitemaps by hand. I've never bought into the "feature" of running multiple sites from the same code base. Where are you going to put multiple robots.txt files?

Dan Dan says:

eep!- one minor correction:
You wrote:
"Sitemaps: http://mysite.com/sitemap.xml"

Google no likey.

It appears that the line should read "Sitemap" not SitemapS (singular, not plural)

Cheers!

Tom Kraak Tom Kraak says:

Good catch Dan. Thanks.

Website Marketing Website Marketing says:

Great stuff, very useful.

Bruce Chapman Bruce Chapman says:

Tom I keep running into your blog on Google in my daily trawl so you must be doing a good job of SEO. If you weren't aware, I have a free download for generating Blog sitemaps. I developed an extensible provider-model Sitemap generator before there was a sitemap.xml in DotNetNuke. The design I have implemented allows module developers to extend the base sitemap, so that modules like the blog module can have an entry generated for each separate Url in the blog. You can find it at http://www.ifinity.com.au/Products/articleType/ArticleView/articleId/3/

Tom Kraak Tom Kraak says:

Thanks Bruce, looks very promising. I'll take a closer look at your sitemap provider shortly.

Lugaluda Lugaluda says:

Nice... thanks for sharing this tip dude... I'm currently having problems on google crawling my web pages. Will give this a shot...
Thanks....

Comments are closed

Subscribe to our Feeds Follow on Twitter