A sitemap is the file that you specify to let the search engines know about the pages, videos, and images hosted on your website. By creating a sitemap, you can specify which pages are important, when they were last updated, and how often they were changed as well as alternate language versions (if any). You may also create an HTML version of the sitemap for your visitors to allow them to easily navigate your website.

The sitemap should include the following information about the files hosted on your website.

• List of pages, videos and images that should be crawled and provides canonical version of each page.
• Must provide a fully qualified URL (absolute path), and never use a relative path or shortened somain name. If you're redirecting every page to www, non-www path should not be used in the URL and vice versa.
• Sitemap should not include GET parameters such as session IDs in the URL that doesn't distinguish origin file.
• Provide an alternate language version of the URL if there is one.
• Sitemap file must be encoded in UTF-8, and and URLs should be escaped appropriately.
• Only display canonical URLs if there are duplicate pages.

You can create a sitemap in various formats, but HTML and XML formats are the most popular. You may create a sitemap manually, or use third-party tools to generate one for your site. A sitemap should be limited to 50MB and 50K URLs. If you have larger and more entries, you'll have to break them up into multiple files with a sitemap index file containing the list of all sitemap files.

You may specify extended sitemap syntax for video and image files. For videos, you can specify running time, category, and age rating. For images, you can specify the subject matter, type of image, and licensing information.

Do you need a sitemap?

Google and most search engines can generally discover all files hosted on your site if pages are interlinked within the website. You may not need to create a sitemap if (1) your site has less than 500 pages, (2) your site has internal links to every file on the site, and (3) you do not have video, images, and news pages that require additional details.

If you're using an open-source CMS such as Blogger and Wordpress, the sitemap may be automatically created and submitted to Google on a regular basis. You may configure your CMS to autogenerate and auto-submit to your liking. Similarly, some site builder platforms like Wix, Weebly, SquareSpace, GoDaddy, and Shopify will autogenerate sitemap files for you, so you do not have to manually generate them.

Google suggest that you have one and submit your latest sitemap to Google Search Console if you wish to let Google crawl your site ahead of their regular schedlue. You may resubmit your sitemap file to Google as often as possible following the same process you used to submit it first time.

If you have a large website with thousands of webpages that are not internally linked or have many rich media files, it is generally a good idea to create a sitemap file. Having a sitemap does not guarantee all entries in your sitemap will be crawled and indexed by search engines as they rely on complex crawling algorithms to schedule crawling. However, having a sitemap will never be a bad idea as it will help search engines like Google crawl your website more efficiently.

Sitemap Generator Third-party tools

You do not have to create a sitemap manually each time you create a new file. There are many sitemap generator tools available online that are free to use. This is assuming that all of your pages are internally linked, and non-linked pages will not be discovered by the sitemap generators. The following are a few free versions that you may be able to use:

• XML-Sitemaps. XML Sitemaps also provide a PHP script that you can use to generate a sitemap yourself.
• Xml Sitemap Generator
• Check Domains

There are also paid versions of XML sitemap generators such as Slickplan, Dynomapper, and Writempas that offer drag-and-drop, customizable, and more feature-rich interfaces.