Google has announced the completion of a new web indexing system called Caffeine on June 8, 2010. With a whole-new algorithm, Caffeine can provide 50 percent better results than its predecessor. Also, it has the largest collection of web content that Google has ever offered before. Users can easily get the relevant content for their search queries, no matter what they are looking for.
Previously, when a search query was entered into Google’s search field, the result pages were not from the live web, instead they were taken from Google's index of the web. This system does not meet the changing trends in Internet. Today, the web is bombarded with content. The web content is becoming more complex with videos, news, images, real-time updates and more. Online users are also looking for fresh and real-time content. The rich online content and searchers’ need for latest and relevant content have contributed to the invention of Caffeine.
The old search index system had several layers, some of which were refreshed at a faster rate than others. But, the main layer was updated once in every two weeks. In order to refresh a layer of the old index, the entire website was analyzed and there was always a significant delay between when a new page was found and when it was made available for users.
On the other hand, Caffeine analyzes the web in small portions and updates the search index continuously. Any new page added to a site or any update in content in the existing pages is immediately identified and indexed. Now, Caffeine can get fresher and better results for users.
Caffeine indexes web pages not only at a faster rate, but also on a larger scale. It processes hundreds of pages simultaneously and takes up nearly 100 million gigabytes of storage in one database. Moreover, new information is added to the existing database at the rate of hundreds of thousands of gigabytes per day.
The advent of Caffeine will take the search experience to the next level. This robust search index system not only yields fresher results, but also helps build a comprehensive search index. Google has also planned to make improvements to Caffeine in the future.