The DNN XML Sitemap Dilemma

First time here? You may want to check out the blog archive, subscribe to the RSS feed, sign up for free email updates, or follow me on Twitter. Thanks for visiting!

The DNN XML Sitemap Dilemma

Posted on Tue Jul 07, 2009 by Tom Kraak
In: DotNetNuke, SEO, DNN Module Reviews | 13 Comments

A great deal has been said about DotNetNuke’s implementation of the XML sitemap standard, but as far as I am concerned, not much has been accomplished. Some community members lost sleep over minor shortcomings such as the priority bug, which is hailed as a new feature in DNN 5.1, even though Google could care less about it. It’s not that I don’t appreciate the effort that went into making the priority a page setting, but this should have been handled by a core admin module on a single page or automated altogether.

The real issue of not including dynamically created pages, however, remains a problem to this day. Take a look at seablick.com/sitemap.aspx for instance. As you can see, only “conventional” DNN pages such as Home, Blog, About and Contact are included. My entire blog on the other hand is not represented at all. I know of very few real-world DNN sites that manage their content entirely via a combination of pages and text / html modules. Who doesn’t run a blog, article repository or knowledge base these days? And it’s for those types of sites that DNN’s default sitemap.aspx is utterly useless.

Alternatives

iFinity Google Sitemap Provider

I believe iFinity’s free sitemap provider is the most widely used alternative to sitemap.aspx. Since it’s a provider rather than a full-fledged module, installation can be tricky for the uninitiated, but the feature-set shows the direction that the DNN core should be taking. Besides the base sitemap provider, the solution ships with extensions for the core Blog module, the core Forum module, Ventrian News Articles, and iFinity’s own Tagger module. This ensures that dynamically created pages by these modules are included in the XML sitemap generated by the provider. The source code is freely available and includes detailed instructions on how to create extensions for other CMS modules. On DNN 4.x installation, you’ll face the priority bug again, whereas on DNN 5.1, it picks up the priority specified in Page Settings. The provider also includes options for caching the sitemap output to maximize server resources and the ability to generate a sitemap index file for large websites.

Inspector IT DNN SiteMap

This “sitemap on steroids” module by Inspector IT acts as a traditional HTML sitemap, feature-rich DotNetNuke page (tab) manager, as well as XML sitemap generator. I like the fact that it calculates page priority based on menu structure and that it provides configuration options on whether to include optional tags such as <lastmod> and <changefreq> or not. According to Antonio Chagoury, the imminent 4.0 release of the module will generate “virtual Urls” for the DNN Blog and Ventrian News Articles. Well worth the $49.95.

Future

How would you like to see DNN’s XML sitemap implementation evolve? Should DNN Corp put more resources into this area or is it better handled by 3^rd party extensions? Are base modules / providers plus add-ons the answer or should the ultimate solution be crawler-based? I’m sure you have an opinion, so please chime in!

Comments

Wednesday, July 08, 2009 at 6:42 PM Rodney Joyce says:

Hey Tom,

Just wanted to mention that all the Smart-Thinker modules implement the Ifinity Sitemap provider so it crawls your profiles, groups, events, leagues etc:
eg. http://www.pokerdiy.com/GoogleSiteMap.axd

It also handles the Priority based on when a profile/group was last edited ;)

I highly recommend this module and all module developers should have a look at it as it only takes an hour or 2 to implement...

Wednesday, July 08, 2009 at 7:37 PM mamlin says:

Great post, Tom! Even as one of the community members who "lost sleep" over the page priority bug (really that had to do more with pacifying my Google-webmaster-stats-reading customers), I agree the lack of representation of dynamic pages in the core sitemap is a much, much larger issue. I expect that such obvious holes will gain more focus and attention from the core team in order to help justify the "Pro" part of "Professional Edition". If the DNN team can implement workflow then surely it can also implement a better sitemap.

Instead of roadmapping wish-lists out to infinity, I think the corp should consider trying to immediately incorporate certain free modules / providers into DNN as defaults (ala AVCalendar a few years back) or at least as easy-to-select options. iFinity's Sitemap, Google Analytics and FriendlyURL offerings would be great additions. They're already offered for free and, if iFinity were allowed to retain some sort of branding on those products, might even be free to the corp (to use/incorporate but not to own). The corp would be offering module vendors a very direct way to get good, free products into user's DNN installations so I think cost to the corp should be nothing. The real trick would be in determining a fair and impartial means of deciding what modules/providers need alternatives and which are allowed in...

Thursday, July 09, 2009 at 2:54 PM Tom Kraak says:

@ Rod - thanks for adding that info.

@ Mamlin - your point about core adoption of free modules is a good one, but I don't remember seeing that happen once over the years.

Thursday, July 09, 2009 at 5:51 PM mamlin says:

The closest I recall to core integration of a free third party module was the AVCalendar I mentioned. That really only happened because Alan Vance joined the DNN team to rework his AVCalendar into what was launched as the EVENTS module replacing the old DNN CALENDAR module.

One brief history is provided here:
http://mydnn.org/Default.aspx?tabid=200&EntryID=5

Friday, July 10, 2009 at 6:13 PM Adam Paxton says:

I noticed that after I started using Ventrian NA's new 'shortened URL' option, the ifinity sitemap provider still serves up the longer URL. Is that happening with you, or are you shortening it with urlmaster and regex?

Saturday, July 11, 2009 at 3:33 PM Tom Kraak says:

@ Adam - I'm running Url Master with custom rewrite / redirect rules in FriendlyUrlParams.config and a "manual" sitemap.xml.

You bring up a good point though, as this is one of the challenges of the "base modules / providers plus add-ons" approach. In this case, Bruce Chapman (iFinity) has to keep a close eye on what Scott McCulloch (Ventrian) does and keep up with changes accordingly.

On a slightly different note, be careful switching to NA's new "shortened Url" mode for an established blog / article repository as there is currently no build-in 301 redirect functionality.

Thursday, July 16, 2009 at 12:39 AM Bruce Chapman says:

@Adam : there's a new fix for the shortened News Articles urls. I've just got one more thing to fix up and I'll post it for download. This also includes the DNN forums provider - Tom jumped the gun on that one :-) Scott M is pretty good at letting me know these days if there is a breaking change, and it only takes a few hours before I find out from other people anyhow!

@mamlin : while I wouldn't mind if the products got included in the standard build, I'm sure the legal/licensing issues are really the blocking factor. That, and if every DNN user hit me up with some support questions I'd soon drown in emails. I've offered to turn over ownership of a couple of things in the past, but nothing has ever happened. It makes little difference to me, I'm just happy to put them out there and have people find them when/if they need to. In any case, I'm sure the core team are gradually going to build in the features over time and make mine redundant - although I always like to be a couple of features ahead of the game.

@tom : you can be brave and use the NA sitemap provider. It will work with your custom url scheme for this blog.

Thursday, July 16, 2009 at 9:18 AM Tom Kraak says:

Thanks Bruce, I've tested your NA sitemap provider with my custom Url scheme and it works fine. Will switch soon.

Thursday, July 16, 2009 at 2:18 PM Adam Paxton says:

Thanks Tom and Bruce.

@Tom Thanks for the note on the 301 issue. I've been following that conversation, even though I went ahead and switched over to shortened urls. I've got a custom rewrite as well, but can't seem to get a custom redirect to work. My regex mojo is lacking.

@Bruce, I'll keep an eye on your site for the latest version of the sitemap provider. I was on NA 7.45 when I posted the note to Tom, since then, I've upgraded to 7.46 to fix an RSS bug, but I've noticed that google sitemap provider has stopped working for me completely, returning a 404. This only happens to portals with NA loaded.

I'll wait for your update, and if I still have problems, I'll post on your forum. Thanks again.

Friday, July 17, 2009 at 12:13 PM Mitchel Sellers says:

@tom - Hey - Once you get the updated site map working with Bruces items....let me know....

Friday, July 24, 2009 at 4:41 PM Declic Video FX says:

Hello Tom,

Will you switch to iFinity Sitemap ? On my side, I still have some problems with News Article (maybe because my website is bi-lingual). We discussed the topic with Bruce, but so far it seems that there is no solution. :-(

Do you have any other solution to provide (or if Bruce has any answer to my posts on his forum) ?
DV FX

Friday, July 31, 2009 at 3:28 PM Tom Kraak says:

@ Declic - yes, I'm in the process of switching to the iFinity Sitemap Provider.

Sorry, I don't have DNN ML experience at all.

Friday, September 04, 2009 at 1:15 AM Ben says: