There are many different ways to get a page to index. We wanted to test to see how long different indexing methods take. As there is some doubt as to whether indexing works at all in the first place, the series of tests below will demonstrate whether or not indexing is a viable technique to get Google to crawl your pages. The following methods were tested: Google URL Submit, Search Console’s Fetch and Render, Linklicious, and Published to the web in a Google Doc.
Test Setup
There are five tests within this test. After a page was published, it was immediately subjected to each specific indexing method. The page was tested to see if it appeared in search results and checked for a cache date.
Test One: Linklicious
The target URL was submitted through Linklicious’ dashboard. It was submitted on September 20th. The target URL indexed on September 28th.
Test Two: Google Doc, Hyperlinked
A Google Document was created and on the Google Doc was a clickable link to the target page. The Google Doc was published to the web on September 20th. The target URL indexed September 30th.
Test Three: Search Console, Fetch and Render
The target URL was submitted through Fetch and Render. It was submitted on September 21st and indexed on September 21st.
Test Four: Google URL Submit
The target URL was submitted through Google URL Submit. It was submitted on September 21st and indexed on September 21st.
Test Five: Google Doc, not hyperlinked
A Google Doc was created. This time, the link to the target URL was not clickable. It was just text. The target URL was published on September 14. It’s been a month and the page has failed to index.
Results
The non-hyperlinked Google Doc is an important piece to this test. It demonstrates that the methods actually work. It simply isn’t time that allowed for the pages to index. Additionally, it’s an important point to note that if you are using the Google Doc method you must use clickable links in the doc. If you are having difficulty getting a particular page to index, and are adverse to using Search Console, we recommend using the Google Doc method. We think it is a more natural indexing method than pinging Google using a link indexer.
Clint’s Feedback
In this video, Clint discusses this test, his experience with Google Search Console, Mobile Friendly Test, and indexing.
Test Number 66 – How Long Does It Take For A Page To Index Using Different Indexing Methods?
I think this was a 2017-2018 test and it’s certainly one that needs to be looked at, again. But here’s the different methods that we’re used to index. One is Linklicious, I remember that service. I wasn’t a fan of it, but I remember it. The old Google Doc – you’re linking to Google Doc method, the Search Console fetch and render, not today, it’s URL inspect submit to indexing, Google URL submit, which is again, the same thing. Google Doc, not hyperlinked and there’s the times right there.
URL submission was less than 15 minutes, Fetch and Render was less than 15 minutes, Linklicious, it was eight days, that’s better results than I got with it ever. Google Doc hyperlink was 10 days, and then Google doc not hyperlinked was never.
Now we know today that those are all completely different. URL submit was depreciated, and then put into Google Search Console, as part of the URL inspection tool. So the fetch and render is the URL inspection tool now. And then you do the test live version, and that fetches and renders the new code versus what’s in the system stored currently. And then you hit the Submit to index or request indexing button, and that replaces the URL Submit.
As you all know, if it isn’t that lately, Google Search Console is running on a different database, it has to be running on a different database than what the search results are because there are times when the Search Console will say, your pages are indexed, you go look into the SERPs to look for the URL to see that is indexed or not, and is not there or vice versa, you find it inside of the search results, but the search console is telling you that it’s not indexed.
That is becoming more and more and more of a problem, because they’re trying to tie those two databases together and then trying to tie some features together. And in particular, the PageSpeed Insights, or core web vitals, and the mobile indexing or the mobile friendliness tests.
They’re trying to make those all so that if you use the search console, you have the mobile friendly test, for example, and you fail it, that will remove you out from the SERPs. And the same with core web vitals, I expected that to happen too if it’s significantly bad and your competition is significantly good, then you won’t get taken out of the SERPs, but it may affect your ranking. Mobile for sure, because they’re driving towards that mobile only indexing, we’re going to index the mobile version of your website right? They keep saying that switch over keeps getting pushed down the road because most people are building responsive websites anyway and everything loads in the T version.
Mobile only indexing in my opinion, would have worked great when you had to use that whole .mobile and there was no such thing as responsive web design, it was all two versions of your web page. And now to me, it just kind of outdated. But the mobile thing, they want all the websites to be mobile friendly and that’s really about revenue, right? If you do a mobile friendly website and the majority of the internet traffic across the world is mobile friendly, or is mobile traffic, then they figure that we’ll adjust this larger customer base by saying – hey, everyone’s website should be mobile friendly and if you’re not mobile friendly, you’re not going to be in our index, because we’re using mobile users and we’re doing our CTR and CRO stuff to Google SERPs, based on mobile users and we only want to show them sites that they can open up inside of their mobile thing. And that will increase our value as a search engine to mobile users, ergo, they will click more of the mobile ads.
It’s nothing to do with making the internet better. It’s all entirely to do with the conversion rate optimization of Google and the search results. So when they plug in a broken tool, like the mobile friendly test tool, into Google Search Console, and then if you fail the mobile friendly test tool, Google search console signals the SERP database, and that removes your ranking from keywords across the board based off of data of the mobile friendly test tool, it’s not going to be helpful to webmasters at all. It’s something I’ve seen more and more consistently, is that the mobile friendly test tool will show you a fail and then if you had any rankings on that page, all those rankings with disappear.
The page will still be indexed, but you lose all the rankings and that’s because of the technology and the mobile friendly test tool is not up to the standards of the technology that’s being used today. And ergo, it is causing false positives or false negatives in this case, and sending that signal to search console and search console sending it to the SERP database, and that is messing up webmasters left and right.
At the end of the day, Google could give two shits less about that. They’re not going to fix it. If there’s a major publishers, they get caught up into that mix, and they have a significant business relationship with Google, maybe they they’ll address it. But for the most part, people are just going to have to bend to the whims and means of what the limited amount of tools that they give us, Google’s giving us, in regards to being up to date and working properly across the board for all websites, versus fixing it and making sure that everything works perfectly, though they’re totally fine with outliers failing, as long as it works for most people.
So keep that in mind, keep an eye on that, in particular, the mobile friendly test. Look at it, if you have pages pop in there, check the rankings if they’re gone. Figure out the mobile friendly thing. Totally from my experience, you’ve corrected a mobile friendly thing, and then the rankings come back, like almost within 24 or 48 hours. If you ignore the mobile friendly thing, the rankings will stay gone until you fix it. And then once you fix it, they’ll come back. And I’ve seen it time and time and time and time again, there’s a lot of people who say, I’ve never seen that. There’s not a lot of people who’ve actually bothered to look and check its rankings.
Indexing stays. If you fail the test, it’ll stay in the index. It just won’t rank for anything.