The ultimate goal of any SEO strategy is to dominate the top of the search engines. Before this can even begin to happen, though — and way before on-site optimization and inbound links become a factor — your website needs indexing.
Indexing simply means that your content is recognized by the search engines and can be discovered via relevant search queries. If organic search engine traffic is an important part of your long-term strategy, being regularly indexed by the likes of Google, is essential.
Of course, indexing your website requires a visit from the infamous Googlebots. These robotic critters ‘crawl’ your site, searching every nook and cranny for new web pages. When new (or updated) content is detected, the URL is passed back to the search engines to be indexed.
Once indexed, your site will start appearing in the SERPs. Now your SEO strategies can begin to take hold, helping you to rank higher.
In other words, getting indexed is SEO strategy step one.
The easiest way to get indexed is to build an XML sitemap for your website, and then submit it to Google Search Console. Luckily, this is easy for WordPress users, thanks to plugins such as the excellent Google XML Sitemaps plugin.
(Note: If you’re wondering whether to use this particular plugin and/or Yoast’s rather excellent WordPress SEO plugin, take a quick look at this earlier article of ours on exactly that.)
The Google XML Sitemaps plugin, created by Arne Brachhold, has been available for the best part of a decade. You can download it for free from the WordPress repository, where it has the distinction of being the top-rated WordPress plugin of all time — with a 4.9 out of 5 rating and an absolutely whopping 18 million-plus downloads to date!
Today, I’m going to be showing you how to install and configure this superb plugin. Let’s get started.
What Is an XML Sitemap?
First, let me answer the question: What is an XML sitemap?
A sitemap provides an overview of your website’s structure. It’s created to help search engines uncover all of your live content — including new URLs and recently updated posts.
The XML part refers to the markup language used to communicate additional information to the search engines — such as when a page was published, or when it was last updated.
By following your sitemap, search engines can crawl and index your entire site — unless you choose to attach a noindex tag to a page. This means a sitemap is the only way to guarantee that all (required) content hits the search engines.
Consider the alternative: With no sitemap, search engines have to stumble across your website, possibly by chance, or, more likely, as a result of inbound links.
The crawlers may still index some of your pages this way, especially if your site is well organized with lots of internal links. However, there is no guarantee that every post and page will be indexed. Indexing is likely to happen far more slowly, too.
How to Install Google XML Sitemaps
Now we’re on to the meat of the post: How to install and configure the Google XML Sitemaps plugin. Don’t worry if you haven’t installed a WordPress plugin before — the process is remarkably simple.
Log in to your WordPress dashboard, then make your way to Plugins > Add New. Next, type ‘Google XML Sitemaps’ into the search bar. There are lots of similarly named plugins, so make sure you get the right one (you’re looking for the one by Arne Brachhold).
When you find it, click Install Now > Activate Plugin. Job done.
As soon as you hit activate, Google XML Sitemaps will generate your first sitemap automatically, taking a snapshot of your website in its current state. You can view your sitemap by adding the /sitemap.xml extension to your domain.
Your sitemap will list all of your URLs, as well as priority level and change frequency (more on these later). For now, though, here is a screenshot of my (admittedly basic) XML sitemap.
How to Configure Google XML Sitemaps
Now it’s time to start configuring your new plugin. All of the plugin’s settings are controlled from one screen — found by navigating to Settings > XML-Sitemap.
Fortunately, the plugin comes with a sensible set of pre-configurations — you could probably leave the settings as they are and the plugin would work just fine out of the box.
Personally, though, I prefer to learn a bit more about what my plugin can do, so that I configure it optimally based on my requirements. If you like the sound of this, read on, as this section covers all the main settings.
Straight after installation, you may see the following message at the top of the screen: ‘Search engines haven’t been notified yet.’
If you want to get your site indexed as quickly as possible, hit the notify search engines about your sitemap link.
This will ping your sitemap to the two major search engines: Google and Bing. Seconds later, the plugin will (hopefully) tell you that your ping was successful. Awesome — you can now expect a visit from the Googlebots, and your content will soon be indexed.
Now back to the configurations screen, which is split into seven sections. Let’s go through each section in turn, giving you a full rundown of what the plugin can do.
1. BASIC OPTIONS
As the name implies, the first section controls the basic configurations.
For starters, you can decide which search engines you want to notify about your site’s updates. My recommendation is to leave both the Google and Bing checkbox ticked. Every time you publish a new post or edit old content, the ping process is repeated, and the crawlers are sent to scope out the changes.
The other option adds your sitemap to the robot.txt file. Now, robot.txt is a file specifically created to communicate with search engine crawlers — it guides them around your website and tells them what to index. The file is especially useful for search engines that don’t support ping notifications (usually only the smaller ones). As such, I recommend leaving the robot.txt checkbox ticked, as this can get you indexed in some extra search engines.
Scroll down, and you’ll see the Advanced Options. The first two settings regarding memory limit and execution time are largely redundant, so go ahead and skip these.
Further down, you’ll be asked to configure the XSLT stylesheet. Remember the screenshot of my basic sitemap I showed you earlier? It wasn’t the prettiest, right? Well, that was the prettified version of the sitemap (gasp!). It wasn’t the easiest on the eye, but it was still possible for a human to read.
You can choose to upload a new XSLT stylesheet if you want your sitemap to look nicer, or you can disable XSLT altogether if you want it to be less ‘readable’ (don’t worry — search engines will still read it just fine).
The next option is perhaps the most important in this section — override the base URL of the sitemap.
Now, by default, your sitemap will appear at the /sitemap.xml extension — e.g. domain.com/sitemap.xml.
But let’s say you split your website across sub-directories.
- Root domain: domain.com
- Sub-directory: domain.com/blog/
If you wish to combine all sitemaps in the root domain sitemap, enter the sub-directory URL in the blank field. Now, you’ll need to dive into your root domain’s .htaccess file and paste the following code:
RewriteRule ^sitemap(-+([a-zA-Z0-9_-]+))?\.xml(\.gz)?$ /your-blogdir/sitemap$1.xml$2 [L]
Note: In the above code snippet, be sure to to replace ‘your-blogdir‘ with your own sub-directory name.
The final field configures an HTML version of your sitemap for less sophisticated bots that can’t understand XML. This won’t apply to any of the big search engines, but, if you want to be indexed everywhere, leave the box ticked.
2. ADDITIONAL PAGES
This second section, Additional Pages, is only really applicable to those using sub-directories and sub-domains — it lets you specify additional external pages to add to your sitemap.
For example, if you run your blog at domain.com/blog/ you may still want to include the homepage, domain.com, in your sub-directory sitemap.
Just click Add new page and follow the simple instructions. You’ll need to specify the URL, priority level, change frequency, and the date the URL was last changed — we’ll be explaining priorities and change frequencies in more detail further down the page.
3. POST PRIORITY
Next, we have Post Priority. To help you better understand this section, allow me to quickly introduce SEO priority levels.
Now, priority levels are a vital part of your sitemap — they communicate your most important content to the search engines. Every individual web page gets its own priority level, scored between 0 and 1 in increments of 0.1.
You might think: Well, my site is super-important, so I’m going to give all my URLs the maximum priority of 1.0.
A URL’s priority level is judged relative to other internal URLs. If you give them all the same priority — even maximum priority — it communicates to the search engines that all URLs are equally valuable. And do you really want your homepage prioritized to the same extent as an obscure blog post? Of course not. So have some discipline, and be sensible when assigning priority levels.
Back to configurations; this section looks specifically at how each post’s priority is scored. There are three options:
- Do not use automatic priority calculation — this treats all posts equally, with the actual priority value assigned later.
- Comment count — priority is calculated based on the absolute number of comments a post receives.
- Comment average — priority is calculated based on the number of comments relative to other posts.
My tip? Use one of the comment-based calculations. This means your popular posts will automatically receive a higher priority level.
4. SITEMAP CONTENT
The Sitemap Content section allows you to choose which items, custom post types, and taxonomies you want to include in your sitemap.
For example, the homepage is an absolute must. It’s also a good idea to include all of your WordPress posts, pages, and custom post types. However, personal preference will dictate whether or not to include your archive or tag pages. If you don’t want these pages appearing in the SERPs — and the value of these pages is questionable at best — make sure they’re deselected.
At the bottom of the section, you’ll see Further options. This will ask if you want to include last modified times. I recommend leaving this ticked, otherwise the search engines will only crawl brand new content — not recently edited content!
5. EXCLUDED ITEMS
You may want to include most of your blog posts in your sitemap, but not all of them. This section allows you to exclude entire categories from the sitemap, plus individual posts and pages by entering their ID numbers.
6. CHANGE FREQUENCIES
We’ve mentioned Change Frequencies throughout this post, but this is the section where you can define values for them.
Now, change frequencies are guidelines given to the search engines, letting them know how often content is likely to change. This information tells the search engines when they need to send their crawler bots to look for updates.
For example, if you publish blog content regularly, set the homepage change frequency to reflect this — perhaps daily. If you run a static website that rarely changes, daily feels like overkill. In this scenario, the yearly option feels more realistic. The options are always, hourly, daily, weekly, monthly, yearly, and never.
Remember, though: This isn’t an exact science, so don’t expect the bots to arrive like clockwork. However, the change frequencies communicate clearly how often a type of content is updated relative to the rest.
We’ve already looked at Post Priority levels, but this section lets you rank how important your other content is — you must also specify blog posts’ maximum and minimum values.
Your homepage is probably your most important page — so go ahead and give it the maximum score of 1.0. Static pages and popular blog posts are likely to be your next most important pages, so score them highly, too.
Just remember: don’t score unimportant content too highly. This dilutes the priority levels of your truly important stuff.
And with that, we’re done: An in-depth look at how to configure the Google XML Sitemaps plugin.
The plugin is free and comes with a big reputation — top-rated in the WordPress repository. This is probably because the XML sitemap functionality is invaluable, plus the plugin is easy to set up.
One of the plugin’s major strengths is that the default configurations are sensible. This means you can simply plug in and play — the stress and hassle-free option! However, for specialist SEOs, the plugin has loads of settings to play around with (as demonstrated in today’s post), so you can tweak your sitemap to your liking.
Overall, a truly exceptional plugin — and arguably one of the very best WordPress sitemap plugins around.
Using Google XML Sitemaps? Thoughts?