We thought it would be interesting to see if we can inflate the number of searches with some CTR tools and see if that inflation will make any impact on rank. We decided to run a CTR test to figure out if one of the new Google patents comes into play. In particular, this patent from March 12, 2019 (Modifying search result ranking based on implicit user feedback).
Test Downloads
Transcription Below
This test is by Boryslav. Boryslav, for the last couple of months have been running more CTR tests than I think anyone in the planet. I would be surprised to find out that someone is doing more CTR work than he is, and I asked if he would show some of the results. In particular, he has been taking a lot of well-known CTR tools. Most of these are bots or some form of a bot. The ones that experts claim, that people that sell stuff say that this is totally working. He has been playing with them.
Recently, March 12, Google released a patent and that patent was modifying search ranking based on implicit user feedback. The idea is, everybody who follows this kind of stuff is saying this is CTR. This is what it is doing. There is the link. I highly recommend that you check it out and read up on your own.
What he did was set up one random site, this was in the wild, choosing a site that exists, medium competition keyword, and he sent a hundred visits per day for the duration of a couple of weeks using a CTR tool. Lots of it he’s done, we’ve seen value in the test environment when nothing else was happening.
Now we want to throw this into a live environment because we want to see if there is really any value here. In this particular case, this site dropped. This is the start and now it is getting a hundred clicks a day through the CTR tool and you can see the slow descent downward.
There are obviously a lot happening and there are a lot of variables going on. The big take away for this, for me, is if you are going to buy a tool and somebody claims that this thing works, don’t send it right to your client’s sites. Get some real data from these people. A lot of people are selling CTR tools, I am sure, and I am confident that some work. We are having trouble finding that one that is working consistently every time.
Andrew: The experiment here is not testing the patent, it is testing the provider, is that right?
I think that is very fair.
Andrew: I think that other tests, previously, historically, I think I was even involved in one a couple of years ago with Mechanical Turk, have shown that there is definitely some impact from clickthrough rate and use of signals on rank. It’s just on how strong it is and depends on what that traffic looks like.
The only thing that is infuriating, frustrating, about CTR tools is actually finding one that consistently works. I really haven’t seen any yet and I know people have made a lot of claims and a lot of people that I trust. But I haven’t seen a tool that you can consistently say, yes – this works every time. So, that is the kind of thing that, there has got to be some other things in play with CTR tools.
There is a question in the group, what kind of tool this is, we can’t give away this tool up, what this was, in particular. That is not the point of this example, if this tool does or does not work. The point is, put testing methods into place.
This is just like with the PBNs, just because anecdotically, everyone says it is the right provider or everyone says this is the right tool, do the research yourself. Ask them for the research that they have, then do the research yourself to make sure that it does actually do what it is supposed to do before you fire it at your own sites or client’s sites, you invest a lot of money, time and effort, there are small tests that you can do to see where things are at.