Image of a person holding a computer, overlaid with the text 'What is a sitemap and how to create them'
5 minutes

What is a Sitemap and How to Create Them

Creating a sitemap is essential to improve the indexing and crawlability of your website by search engines. Sitemaps are files where information is provided about the pages, videos, images, and other files on your site, as well as the relationships between them. 

Search engines like Google read this file in order to more intelligently crawl your site, and understand what pages you would like to be indexed (you might not have every single page on your website indexed), or to understand when a page was last modified.

That being said, it’s also worth clarifying that Google is increasingly refining its processes, so as Google itself puts it: “If your site’s pages are properly linked, our web crawlers can usually discover most of your site.”

What this means is that while there is a good chance that your site would be fine without a sitemap, it’s definitely something that will help your SEO – and if you’re aiming to have your site dominating search, it’s something that you won’t want to leave to chance. 

As Google relies on links to crawl a site, an example of a website where a sitemap would be even more essential would be a site with hundreds of thousands of pages, or even millions. In instances like this, unless internal linking is faultless (which rarely is), Google would struggle to find all of the pages in your site without a sitemap.

The benefits of having sitemaps

So, despite Google on most cases being able to find all the pages on your site through normal crawling of the site, there are definitely a few indubitable benefits of having sitemaps:

1 – Enhanced Crawling and Indexing 

Sitemaps streamline the process for search engines like Google, Bing, and Yahoo to crawl and index your website’s content. It’s also another opportunity for a site to signal to Google what its key pages are. That’s why a sitemap should only list pages that you want indexed and should be kept as tidy as possible. This ensures that your webpages are effectively incorporated into search engine results pages (SERPs).

2 – Faster Content Updates 

Sitemaps let search engines know about new or updated content on your website, enabling them to crawl and index these changes more promptly than they might otherwise. This helps ensure your website’s search results are always current and up-to-date.

3 – Clear Content Organisation

Sitemaps help offer a structured overview of your website’s content, providing search engines with a comprehensive understanding of the hierarchy and relationships between different pages. This facilitates more effective crawling and indexing.

4 – A better understanding of your site’s performance

Provided your sitemap only has pages that you have decided are worthwhile of being indexed, having a sitemap submitted allows you to analyse in Google Search Console exactly how many pages are or aren’t indexed within all the pages you’ve submitted, and get an explanation as to why each group of pages wasn’t indexed.

Screenshot of Google Search Console indexed pages, showing the list of reasons for no index

The different types of sitemaps

There are several types of sitemaps, but the one that will be more common to most sites is the XML sitemap. The different types of sitemaps include:

HTML sitemaps:

This is more like a content sitemap that users can see and use to navigate your site. HTML sitemaps are increasingly seen as outdated or even entirely unnecessary, and a website with internal linking set up correctly certainly won’t need it. While HTML sitemaps might help users find pages on your site, as highlighted previously your internal linking should take care of that anyways. So the focus from an SEO perspective should be on XML sitemaps.

XML sitemaps:

These are the sitemaps that’re purely used for indexing and crawling your website and need to be manually submitted. It’s the more modern form of handling how all your content is stored across your website.

HRSS/Atom Sitemaps:

These sitemaps are specifically tailored for websites that have frequently updated content, such as news or blog sites. They leverage RSS or Atom feeds to notify search engines about new or updated content.

Video Sitemaps:

As the name implies, video sitemaps are designed to provide information about video content on your website. It includes details like video URLs, titles, descriptions, and other relevant metadata.

Image Sitemaps:

Similar to video sitemaps, image sitemaps function by informing search engines about the images on your website. They include details such as image URLs, captions, and other pertinent information.

News Sitemaps:

These specialised sitemaps are designed for news websites and provide information about news articles, including their publication dates, titles, and other relevant metadata.

The different ways to Create a Sitemap

Option 1: Use a Sitemap Generator Tool

If your website is custom-coded and is not on any CMS or builder that generates a sitemap, you need to use a sitemap generator. There are many tools online that can create a sitemap, when using one some of the factors to consider include:

Ease of use: How user-friendly it is, does it have intuitive navigation and clear instructions, guidelines, or tutorials?

Learning curve: Do you need extensive technical knowledge to be able to use it?

Sitemap types: Does it support various formats for various use cases?

URL limits: Are there limitations on the number of URLs  it can handle? And, more importantly, does it meet your website needs and is it scalable as the content on your website grows?

Customisation: Can you include/exclude specific URLs, set priorities, and update frequencies? (Remember, you only want to include on your sitemap pages that you definitely want indexed).

Accuracy: Does it generate accurate and error-free sitemaps? Search engines will lose confidence on sitemaps that have errors.

There are several sitemap generators that you can use, but we’d recommend two:

TechnicalSEO by Merkle is free and has one where you can upload a CSV file with the URLs you’d like to include. It’s particularly great if you have different language versions of your pages (hreflang tags), as it allows that level of customisations. 

Screaming Frog SEO Spider also has one that I like to use with simple custom-built sites. This is an incredibly useful tool for so many purposes, sitemap generation being just one of them. In Screaming Frog, ensure you are using the spider mode. You can do that by clicking on “Mode” and selecting “spider”. Then type the URL of your home page and let it crawl. When it’s done, click on “Sitemaps.”

Screenshot of Screaming Frog and the option to export a websites sitemap

In order to save the XML file to your computer, tick all the options that matter to your site and click on “export”. Remember, you’ll only want to include pages that you want indexed. Then, upload that file to your server in the root directory of your site.

It’s worth clarifying that both of these tools do not automatically update the sitemap file. Some tools do but are premium, so you pay for that service.

However, you won’t need to deal with any of the above if your website is on WordPress or an ecommerce platform like Shopify.

For WordPress sites, popular plugins like Rank Math or Yoast generate sitemaps and update them when you edit your posts and pages. Shopify, on the other hand, even generates sitemaps automatically, which makes things easier.

How to submit your sitemap to Google

After ensuring your sitemap has been uploaded to the root directory of your site, you’ll want to submit it to Google as well. One way to do it is using your robots.txt file by adding the following line to it:


Another way to do it, and highly recommended, is adding your sitemap to Search Console, which is very simple. Here’s how to do it: 

1 – Sign in to Google Search Console

2 – Select your site in the sidebar:

Screenshot of making a new property on Google Search Console

3 – Access the Sitemaps section

The Sitemaps section is under the ‘Index’ hub. If you do not see it, click on ‘Index’ to expand it.

Screenshot of Google Search Console list 'Pages', 'Video pages', 'Sitemaps' and 'Removals', with 'Sitemaps' highlighted

4 – Enter ‘sitemap_index.xml’ (matching the name of the sitemap you’ve uploaded to the root directory of your site) in the ‘Add a new sitemap’ field to complete the sitemap URL. If you have multiple sitemaps with a sitemap index file, you only need to type the URL for the index file.

Screenshot of Google Search Console highlighting the area to add your sitemap URL

If all goes well, you’ll soon be able to see its ‘Status’ updated to ‘Success’, and also the last time Google has read your sitemap.

5 – Submit your sitemap

How to submit your sitemap to Bing

Bing is the second largest search engine, so it makes sense to also submit your sitemap on it. The process to do it is fairly similar to how it’s done on Google Search Console:

1 – Log in to Bing Webmaster Tools and choose your website.

Image of the Bing webmaster website, with a picture of 3 employees and text 'want more users for your site'

2 – Navigate to the sitemaps section.

In the dashboard, navigate to the Sitemaps section from the left sidebar.

Screenshot of Bing webmaster tool, pointing to the sitemap section

The sitemaps section includes all the sitemaps and/or feeds that Bing knows about your website. If you have already submitted any sitemap for your website or imported it from Search Console, or even if Bing organically discovered your sitemap, it will be shown there. You may also consider removing any old sitemap if it isn’t relevant to your website anymore.

3 – Submit Sitemap

Finally, click the Submit Sitemap button. And in the prompt that appears on your screen, enter the URL of your sitemap in the same way that you’ve done it for Google Search Console.

Importing your Sitemap from Google Search Console to Bing Webmaster Tools

Bing also lets you import any existing verified properties from GSC. So if your website is already verified in Search Console, you can import your site to Bing Webmaster Tools with just a couple of steps. This method can be particularly useful if it’s your first time using Bing Webmaster Tools.

On the Welcome page, click the Import button available under Import your sites from the GSC option.

Screenshot of Bing webmaster tools, with an arrow pointing to the button to import your Google Search Console sitemap

Bing will request your confirmation to fetch data from your Google Search Console and import it. Click Continue. 

Google will then request you to authenticate and then request your permission to let Bing access the Google Search Console data of your verified properties. Click Allow.

Bing will then fetch all the verified websites available in your Google Search Console account, and  you’ll be able to select the sites that you want to import. Select the sites to be imported and then click the Import button.

Once confirmed, you can now double check if Bing has imported your sitemap, by navigating again to the Sitemaps section from the left sidebar, where you should be able to find the relevant sitemap(s).

Sitemap best practices

Finally, these are our top three suggestions in terms of best practices to maintain your sitemap:

1 – Do regular sitemap maintenance checks and updates

Always check your sitemaps regularly. Search console does a great job of letting you know if your submitted URLs have issues with either crawling or indexing.

Check the ‘Coverage’ section in Google Search Console regularly and make sure to update your site or sitemap when there are errors. It’s particularly effective because it tells you what the exact error is with suggestions on how to fix it.

As mentioned before, you can also use Screaming Frog for sitemap maintenance. After crawling your website or sitemap URL, you can check the response code tab for 404 or 5xx errors.

If using an automatic sitemap generator tool or plugin, you’ll need to update it when updates are available, and also periodically check it by going to your sitemap URL and checking if any page is missing or the last updated time is incorrect.

2 – Include only canonical versions of URLs in your sitemap

As highlighted previously, your sitemap should only contain URLs that you want search engines to index. That means if a URL is set to noindex or points to a different URL as its canonical version, you shouldn’t include it, as it’s a statement to search engines that you don’t wish for that URL to be indexed.

Ignoring it and including those URLs in your sitemap provides conflicting information to search engines. The unintended URL might get indexed, or you will get coverage errors in GSC. 

3 – Split up large sitemaps

If you have a particularly large site, you’ll need to split your sitemap into multiple files if it either exceeds 50MB or has more than 50,000 URLs. In any circumstances you should submit very large XML files to Google, otherwise some of your URLs will not be indexed. One quick tip here is to save each file with easy to understand names like page_sitemap1.xml and page_sitemap2.xml.