This week, let’s see how to provide a dynamic Sitemap.xml to help web crawlers index your site.

If you don’t know what Sitemap.xml is it for, it’s a file with a list of pages of a site, used to inform Google and other search mechanisms about the site’s content. Web crawlers like Googlebot reads this file to track your content.

So, the first thing is add sitemaps package  

meteor add gadicohen:sitemaps

Now create a new server side file sitemap.js and use sitemaps to create ‘/Sitemap.xml’ route. This is an example of sitemaps configuration, you can see more on sitemaps page on Atmospherejs


[code language=”javascript”]

if (Meteor.isServer) {

sitemaps.add(‘/Sitemap.xml’, function() {

var posts = Posts.find({published:true}).fetch();

var out = [];

_.each(posts, function(post) {


page: ‘/post/’+post.title.replace(/ /g,”-“)+’/’+post._id,

lastmod: post.updatedAt?post.updatedAt:post.createdAt



return out;





Now if you access http://localhost:3000/Sitemap.xml you’ll see something like that


[code language=”xml”]

http://localhost:3000/post/Post-0/J2AXrBpeGwkhsH8Kp 2015-04-18T21:09:17+00:00 http://localhost:3000/post/Post-1/f8DkaDyW2S5Zag6cM 2015-04-18T21:09:30+00:00 http://localhost:3000/post/Post-2/PFFLjg7p4ShFLHfRQ </urlset> [/code]


Ok, now you have a Dynamic Sitemap.xml, if something new has been created, it’ll be there.

The next step is to help web crawlers to find you Sitemap.xml, to do that you need to create a file named robots.txt in the root of your site, the url needs to be something like that http://localhost:3000/robots.txt and the content, something like that:


Now, when the web crawler visit your web site, it’ll find your file robots.xml that has a Sitemap map, pointing to you dynamic Sitemap.xml

Ok, just it? It depends, maybe you can help a little more. Using Google Webmaster Tools you can inform Google about your Sitemap.xml and gain even more time.


Sitemap.xml is useful for many reasons like if your site is really large, if it has a large archive of content pages that are isolated or well not linked to each other, your site is new and has few external links to it, if it uses rich media content or if you just want to anticipate indexing. So, do that, use 10min of your time  and help web crawlers to help you.

Allan de Queiroz

Allan de Queiroz
London based software engineer

XServer forward from Linux text mode for Headless purposes.

Hello, this post is about XServer forward from Linux text mode, **not ssh forward, anything related to VNC** or things like that.Recently...… Continue reading