Why we need an open European search index.
Big Tech controls search indexes, and thus the flow of information. This poses a threat to innovation, freedom of expression and democracy. Let's dive into what is search indexing and how we can fix this issue!
How do search engines work?
Some of you older readers might remember a time when search engines were still fighting it out to determine who would be king. Those days are long gone and now Google is regularly featured in the legal spotlight for its monopoly status. While this discussion is familiar to many, there is even less spoken about the limited total number of search indexes being used around the globe.
Search engines work by first scouring the publicly accessible internet with web crawlers that follow and map links between pages. These pages are then added to a search index. When you enter your search in your favorite search engine, that query is checked against the existing search index database. You can think of a search index as a big digital library where you can look for answers to your questions. If the index doesn’t have the page you are looking for, then it will not be displayed in your search results, not even on page 2 of Google’s Search Results.
How web crawlers work to build search indexes.
You can see now that control over what is included in these indexes can have a major impact on the free distribution of information and can lead to censorship. Google, for example, adheres to the Chinese government’s censorship guidelines in order to remain an available search engine within China. This leads to massive amounts of information being made unavailable to Google users located within China.
Who control the world’s largest search indexes?
As of July 2024, Google holds a 91% share on global search. Graph and data provided by StatCounter.
At present there are only a handful of major search indexes used by mainstream search engines. These include Google, Bing, Yahoo!, Baidu, and Yandex. This leaves the majority of global search results in the hands of three countries’ legal jurisdiction, the USA, China, and Russia.
While there are search engines beyond these five companies, many of these alternatives are using the Big Tech search indexes in lieu of building out their own independent index. Search engines like DuckDuckGo and Ecosia, for example, both use Yahoo! and Bing’s search indexes to present their search results.
What threats are posed by search engine centralization?
Centralization of the internet, be it through government intervention or monopolistic practices by Big Tech companies, poses a huge security risk by making other parts of internet infrastructure dependent upon it. Already in 2013, the New Yorker wrote “a staggering percentage of communications flow through a small set of corporations—and thus, under the profound influence of those companies and other institutions.”
In the event of a security incident, outage, or radical political change the lack of an open web index poses a national security threat to any nation not operating their own index. By leaving this pivotal piece of technology in the hands of tech companies who will always prioritize profits, we are placing a dangerous amount of trust into their hands. These indexes act as a critical infrastructure for the status quo of internet functionality.
Why Europe needs their own independent web index.
Out of caution it is crucial that the EU develops its own web index in order to guarantee stabile internet search access without being dependent upon other nations. One such contender is the Open Web Index (OWI). Launched in 2022, the OWI is being developed by research universities across Europe along side non-profit organizations which are working hard to build an alternative to Big Tech gatekeeping across the internet.
The Open Search Foundation goals.
The project is publicly funded for three years and aims to treat “information as public good, with free, unbiased and transparent access…” If search results are limited by a small number of tech companies with competing interests, we cannot trust that they are providing honest and fair access to information. Both Google and Yahoo! have been accused of favoring their own websites and products over those of others leading to lawsuits claiming that Big Tech practices are highly monopolistic.
Local German search engine MetaGer pushing for the development of an Open Web Index
One partner in the development of the European Open Web Index is Suma.ev, the non-profit organization which collaborates with the University of Hannover, to build MetaGer. The MetaGer project aims to offer users a private search alternative to the Big Tech players. Users are able to select which search engines and indexes they would like to query and all results are returned only with a fully encrypted connection. There is even an option to have your results opened anonymously. With full Tor support and an onion site, you can search all major indexes without needing to worry about the invasive tracking of your data.
MetaGer, like Tuta, is located in Germany and thus benefits from the strict data protection laws of the European Union and the German government
Tuta offers an alternative to Big Tech by offering privacy through encryption.
At Tuta we are actively working to create a secure and private alternative to Big Tech email services like Gmail and Outlook. By using the world’s first post-quantum encryption for protecting emails, we are able to offer both free and paid accounts to users across the globe with the goal of enabling citizens to take their privacy back.
Dropping Google is as easy as flipping a switch. Turn on privacy today by signing up for your Tuta account!