August 28th, 2018
According to statistics in March 2018, WordPress now powers around 30% of all websites on the internet. A quick Google search for “a” shows a good example of just how many sites that equates to (as it would return every site in Google’s index with the letter “a” in it).
Of course, that only includes sites that Google knows about. But even with that, we can estimate that circa 7,581,000,000 of those websites would be using WordPress. With that in mind, it is easy to see just how important Yoast’s WordPress SEO plugin is.
As it is both the most used and best SEO plugin for WordPress, you can imagine how many businesses use it on their websites. That would also include their development and staging sites as well.
Yet, there is a “bug” in Yoast, described as a feature by the developers, which can make an SEO’s life far more difficult when it comes to staging sites; the Disappearing Canonical Tag.
What Is The Disappearing Canonical Tag Bug?
When you create a staging or development version of your website, you need to make sure that it is not crawled and indexed by Google. This means that it would be set to noindex across the entire website.
Naturally, you would also put password protection on the staging site to stop visitors who shouldn’t be able to access it from getting to the site. However, human error can always come into play, so whilst the password protection would stop crawlers, if it were to be disabled by accident, they could get in.
This would then lead to the staging site being crawled, which would likely be a direct duplicate of the live site. No one wants to have their entire site duplicated in Google, right? That’s a huge SEO issue waiting to happen.
So, the noindex tag is added as extra security. However, this is where things go wrong with the newer versions of Yoast.
When a page or site is set to noindex, Yoast removes the canonical tag from that page or site.
If it’s set to noindex anyway, then what’s the problem?
Well, from a Google indexation point of view, there isn’t a problem. But then again, that’s not really what we are looking at, is it? The problem only really comes into play when your SEO is testing the on-page, technical optimisation of the site before it is pushed to live.
As canonical tags are one of the most important aspects of on-page, technical SEO, you need to ensure that they are both implemented on the site, and done so correctly. This involves crawling the entire site using one of the various SEO tools available. However, when you crawl the site, you won’t get any information back about the canonical tags.
Why? Because thanks to Yoast’s new setup, they don’t exist.
This means that you can’t check that the canonicals on all pages are correct and as they should be before pushing the site live! In turn, this could lead to issues with the canonical tags being missed, creating SEO problems on the live site.
How do we solve this?
To be honest, this isn’t really a 100% fool-proof way of solving the issue. For example, you could simply not set the staging website to noindex, but that removes a vital level of security against accidental indexation. Or you could create a robots.txt file to disallow access, but then you still need to modify that robots.txt file when sorting out the technical SEO, which creates another issue.
Yet another way to try and work around the Disappearing Canonical Tag issue would be to create a local version of the website for staging purposes, housed on your computer rather than the internet. This prevents all security and indexation issues when working on the site, but means that you need very strict version control between developers, or you risk them overwriting each other’s work.
The other issue with a local staging version is that you’ll still need a pre-launch testing area that other people can access for QA testing. So, when the site is pushed from local to the pre-launch testing area, it would need to be changed to noindex again for security and indexation purposes, meaning you are back to square one with the Disappearing Canonical Tag.
So what can we do?
Unfortunately, there isn’t much to do other than grin and bear it. Make sure that you put password protection on your staging website, and when you are checking the technical SEO, have your SEO and developer work very closely.
Your developer needs to ensure that the password protection stays active whilst your SEO changes the site to “index” rather than noindex. Then, once the testing is done, your SEO needs to switch it back to noindex.
In other words, a careful and concise process needs to be in place that everyone involved is aware of and following. Every single part of the process needs to be carried out with due diligence, with every cog turning correctly.
If you need help sorting out your SEO processes, or even recovery from website duplication, then why not get in touch with Brave today?