If you keep track of how your website is ranked in Google on a daily basis, you might have experienced a roller-coaster of emotions that come with dropping a few places...
If you keep track of how your website is ranked in Google on a daily basis, you might have experienced a roller-coaster of emotions that come with dropping a few places, bouncing back or even disappearing from the ranking for no fault of your own. The most common explanations for these mysterious movements include rivals staging a successful SEO campaign to surge ahead, a sudden blow to your link profile as some inbound links you have been building get axed or Google introducing changes to its algorithm for greater relevance. Other guesses are also quite frequent – a server error on your side that prevented Google from entering your site during down time or the search engine punishing your property for allegedly breaking some of its regulations.
While all these factors can play a big role in the long-term rank of your website, they are unlikely to be responsible for daily Google ranking fluctuations, both up and down the results pages. What is responsible for these seemingly unpredictable tides is actually Google's web crawling technology.
At the moment, the most popular search engine in the world relies on two databases to return the most relevant results. The primary index is compiled about ten times a year, or every 36 days, by Google's spiders (software which crawls and catalogs pages and their content). The process takes up to a week and its results form the basis for a major index update whose role it to bring the results up to date. It is not an easy leap from the old database to the new one for Google, due to the scale of the operation, so making the latest version public can take a couple of extra days. Needless to say, these loading operations can have an impact on rankings, practically from one minute to the next. They are not normally important for searchers, but companies that rely on traffic generated by their top positions can feel the inconvenience.
The other database that powers results is a temporary index that Google's constantly crawling Freshbot updates on a daily basis. It is on the lookout for new content, as well as other important changes to pages, so that anyone who submits a query can expect not only the most relevant information, but also the latest. This process completes the larger primary index cycle that takes place about once a month. Importantly, the way it operates be blamed for Google ranking fluctuations, too. Since it is a temporary index, Google's spider can write and overwrite information from one day to another. That is exactly why sometimes a page that can be seen in Google one day, disappears without a trace the next. No worries, though. The fact that Freshbot indexed it temporarily is a sure sign that it is bound to be included in the primary index update further down the road.