IndexNow is an evolutionary indexing protocol that can radically change the way content is indexed by all search engines.

Microsoft announced IndexNow, a major evolution in how all search engines can discover and index content. IndexNow is open source and open to all participating search engines. When content is published or updated on a website, IndexNow notifies all participating search engines, dramatically reducing the time to indexing.

The IndexNow sharing between search engines will begin on November 2021. There is a limit of 10,000 URLs per day.

The IndexNow protocol is open to all search engines with a noticeable presence in at least one market.

Evolution of Search Indexing from Pull to Push

There are two ways for search engines to obtain web page data: Pull and push.
Pull is when a search engine crawler visits a site to request web pages and “pulls” the data from the server. This is how search engines traditionally work.

What IndexNow does is changes content discovery to the push method where a CMS like WordPress tells the search engines when the content has been published or updated.

The benefit to publishers is that they no longer have to wait for search engines to crawl and discover the content.

Search engines know about it right away and visit the site to fetch the new or updated pages.

Who is Behind IndexNow

IndexNow was created by Microsoft and Yandex. The protocol is open source and open to all search engines to participate.

When one search engine is notified of an updated or changed web page all participating search engines will be notified.

How IndexNow Benefits Publishers

IndexNow benefits publishers because it will reduce crawling demands on the server, search engines won’t need to conduct exploratory crawls to check if web pages have been updated and it reduces the time for content to be discovered and indexed.

Reducing server load helps the server perform optimally without the added burden of redundant serving of web pages that search engines already have.

Ultimately it benefits the world by reducing the energy demands of crawling and indexing and helping to reduce global warming pressures.

Large Companies are Adopting the New Protocol

Microsoft announced that large companies like “eBay, LinkedIn, MSN, GitHub, and Bizapedia” are planning to migrate to IndexNow from the Bing URL submission API.

It is presumed that other larger companies will follow suit because the benefits of the new system are clear.

Because of that smaller companies may wish to consider also adopting the new protocol in order to stay competitive and improve their indexing.

How Does the New Protocol Work?

There are three ways suggested for submitting URLs.

1. SEO Integration of IndexNow

The first way, OnCrawl and Botify have plans to integrate IndexNow.

2. Direct Submission through API

The second way is a direct submission through an API.

Developers can use an API to automatically submit URLs to IndexNow.

3. Integration with CMS and CDNs

But the third way may be how most sites will interface with IndexNow, accessing the new protocol through a function that is native to the core of whatever CMS a publisher uses.

Wix and Duda recognize the benefit to users and are already on-board with plans to provide the new feature to their users.

Cloudflare and Akamai are supporting the new IndexNow protocol as well.

Waiting for WordPress?

While competitors like Wix and Duda have recognized the importance of IndexNow, WordPress appears to be dragging its feet on the new protocol.

Maybe WordPress is going to adopt it, but for the past seven months the core developers have been discussing the proposal with no apparent resolution.

According to the public WordPress development ticket there appears to not be a decision on it yet.

Nevertheless, Microsoft has submitted the IndexNow open-source code to WordPress for inclusion into the core.

If adopted by WordPress this will make it automatic that a publisher can opt into submitting their URLs via IndexNow.

In response to a question by WordPress core developers, Microsoft disclosed that publishers will be able to choose which search engines to submit their changes and new URLs to.

IndexNow is the Evolution of Search Indexing

IndexNow is a big deal because it’s a major change in how search engines can discover updated and newly published documents.

It benefits publishers with fast indexing and less server load from bots constantly crawling their web pages.

Adoption by major search engines will be an evolution in the relationship between publishers and search engines that benefits both.