Hiya Google spiders,

While you’re here, can you please make sure you check out our main product pages since we make changes to them frequently? Also, we added some new blog pages, so be sure to check those out as well.

Have a great week!

Thanks,

Steph

Wouldn’t it be great to leave a note providing suggestions to search engine spiders about how you want your site crawled when they visit your site? Well guess what? You can.

Spider dog. Courtesy of epicfunnypictures.com.

A XML sitemap is similar to the notes left behind by my well organized sister-in-law when I dogsit for her and my brother. Her notes let me know when to feed the dogs, how often and where to find everything.

A note written for the search engines spiders provides similar details about how to properly crawl your site.

Back in 2005, Google developed a new sitemap protocol in response to the fact that sites kept getting bigger and more complex (foreward ho). The XML sitemap is born. It is designed to make good use of the search engine spiders’ time by enabling the engines to index pages from your site more efficiently.

XML sitemaps do not guarantee that all of your pages will get indexed, but they do help. XML sitemaps also provide you with the opportunity to let search engines know what pages you feel are the most important (i.e. main product or service pages) the last time the content of each page was modified, and how often you expect that content on each page will be updated.

Is Sitemap Submission Outdated?

There are people on both sides of the fence when it comes to the practicality of submitting sitemaps to search engines. One of the reasons why some people are skeptical is because your site will most likely be found by spiders regardless.

In my personal experience the pros outweigh the cons.

As with many SEO tactics, I am a strong believer that every little bit helps. In most cases it won’t have a negative effect to submit your XML sitemap to search engines. So why not?

First of all, it ensures the search engines index your pages. Secondly, you can submit your XML sitemap to one (or all) of the available (and free) webmaster tools provided by the major search engines that help point out crawl errors.

XML Sitemap Creation 101

It is possible to create a sitemap from scratch; however, many people use a sitemap generator and then tweak it. Let’s face it, it’s a tedious task to manually create a sitemap and an automated sitemap generator works.

If you have a static site, try out this XML Sitemap generator.

If you utilize a Content Management System (CMS), there are various plugins that you can make use of for sitemap submissions. Some recommended plugins for CMS’ are detailed below.

Once you’ve created your XML sitemap you can manually play around with the various XML tags and attributes that can be used in a sitemap file, also known as XML schema.

Not quite this manual. Photo courtesy of umanitoba.ca.

The following are the XML schema available and how you can use them for each URL on your website.

• <lastmod>
  • Lets spiders know when you last modified the page.
• <changefreq>
  • Provides information on how frequently the page is likely to change. This is more of a hint though. It doesn’t mean the engines will follow that command.
• <priority>
  • This tag enables you to tell spiders which pages you feel are most important. Main product or service pages are generally given a higher priority. Labeling all pages as high priority will not help you because priorities are relative to other pages on your website.

XML Sitemap Plugins for CMS

If you have a Wordpress site you can try this Google XML Sitemap Generator plugin. It was designed specifically for Google, but it works with the other major search engines as well. A good benefit of this plugin is that every time an update is made to the site, like a new blog post, or a new product is added, the module submits the changes automatically. This can help new pages get indexed more quickly (but not always guaranteed).

For Drupal sites try the XML Sitemap Module. Installing this module makes XML sitemap submission to the engines a breeze. Like the Wordpress plugin, this module submits any changes made to the site automatically to the search engines.

I haven’t had the chance to personally work with a Joomla CMS yet and therefore am not sure which plugin is the best. On the Joomla site this Xmap plugin seems to be the most popular and has the highest number of votes.

If you’re using any other CMS, a simple Google search for XML sitemap plus the name of your Content Management System (e.g XML sitemap Joomla) should provide you with some worthy XML sitemap plugin information.

XML Sitemap. Check. Now What?

If you have a static site (as opposed to a CMS as discussed above) the first thing you need to do is upload the XML file to the root of your domain (add the URL to your site). This will require logging into the FTP with your credentials. The most commonly used format is this: www.yoursite.com/sitemap.xml.

Image courtesy of seraphicpress.com

Once you have uploaded the XML file to your domain, you can edit your robots.txt file to include the sitemap URL. A robots.txt file tells search engine spiders what they shouldn’t crawl. A sitemap tells them which pages they should crawl. Together they compose a detailed note of recommendations you can leave for search engine spiders.

Spiders are programmed to look for the robots.txt file when they crawl your site. As of 2005 you can include a line in the robots.txt to tell the spiders to look for your XML sitemap. If you have multiple sitemaps (for large sites) you can include more than one sitemap per robots.txt file.

You can direct spiders to your XML sitemap by placing this line anywhere in the robot.txt file:Sitemap: http://www.example.com/sitemap.xml

Not having a XML sitemap isn’t the end of the world. In fact, your site can still do extremely well without one.

It is, however, one of the only ways to directly communicate with spiders and leave them a little to-do list. Enticing isn’t it?

My next post will detail how to submit your sitemaps to the major search engines.

——————————————————————————————————

Stephanie Woods is an internet marketer living in Kelowna, BC. You can find her at her (very) new marketing blog . She has also been known to frequent Twitter if you’re into that sort of thing.

47 Responses to “How To Sitemap: Creating XML Sitemaps”