This article was first published as my guest post on AltSearchEngines.com
Perspective
No more than a few generations ago, our grand-grand parents have witnessed the introduction of electricity in every day life. It’s hard for us to imagine today the impact of the first applications of electricity, like the light bulb, had on the people contemporary with Thomas Edison. Then again, this was only the beginning: today, it’s almost surreal, if not impossible to imagine, a life without electric light, radio, television, phones, computers… and, the Internet. What we should keep in mind is that people were living like this not only thousands of years ago, but also less than a couple of hundred years ago.
Comparing to electricity, which, on a traditional evolutionary scale, would be too young to even mention, the Web is an infant. Born less than 20 years ago, it is still younger than the majority of the contemporary human population: a teenager that, for many of us, seems to be around forever.
Today, we think we know what the Web is all about: applications such as Facebook and Twitter define out every day browsing patterns, fill the content of thousands of blog posts and rule our world, despite the fact they’ve only been around for a few years. It is impossible to predict how the Web will look like in 10 years, but we can state using common logic that the Web as we know it today is highly likely to be at least as obsolete in 10 years time as we perceive the way it was 10 years ago.
Humans have a natural tendency to overestimate the present. I think it has something to do with our ego, that leads every generation into thinking they are in some sort of evolutionary peak, that the things they are familiar with will be around, almost unchanged, for a long period of time. The fact is, if we manage to keep things in perspective, we can see how fast and unpredictable the world evolves around us. Many well-established, “too big to fail” industries are struggling to survive and adapt these days, like the written press, the traditional music industry or even the television. Many wrong predictions, business mistakes and false prophecies have as common reason this tendency to overestimate the present and to underestimate the surprises that the future brings us. In that matter, the Web makes no exception.
Trends
The Web has changed a lot since Google first set its mission to organize the world’s information. Back then, the Web had little structure: the information was scattered across all sort of websites and personal pages, none of which looked alike. If you were searching for information on a physics phenomenon
, chances were you would have found the best resources on that particular topic on some obscure page written by a college student for a school project: the type of page you would have never found without the help of a search engine, such as Google.
Following one of the basic laws of nature, the Web, as like any other entity, gains structure as it evolves. Today we are still using Google to search for details on a physics phenomenon, out of habit, but chances are, most of the times, the first result will be an Wikipedia article; we are still using Google to search a person’s name, but again, chances are one of the first results will be that person’s Facebook or LinkedIn profiles. Custom personal pages, so popular and cool just a few year ago, are dying. Publishers are less interested in developing platforms for their publications from scratch and are using one of the few popular blogging platforms instead. Shops are also deployed on existing platforms. Data is moving towards the cloud, new standards have been defined (Atom, OpenID – just to name a couple). These are all telltale signs of the subtle process through which, by emerging patterns, the Web is gaining structure, almost as if it were a living organism.
This evolution comes with a cost. While structure brings initial efficiency, it also brings on new sets of rules, makes the evolving entity become more rigid, “kills the soul” as they say, and, ironically, it’s the first sign of decay. This applies to living beings, companies and industries alike. In 10 years time, it will be very hard, if not silly to gather a small enthusiastic team of developers and work on an innovative web project, hoping it will become a major success; as hard as it would be today to gather a small team of enthusiastic mechanics and being a car manufacturer, hoping to become a name in the industry.
As I have said before, It’s impossible to predict how the Internet will look like in 10 years. Chances are though that it will be almost unrecognizable and that it will have more structure.
Search
The evolution of search is tightly related to the evolution of the Web itself, and particularly to the evolution of the structures that define it. Google has been so successful mainly because it was the first one to take advantage in an efficient way of the most powerful (and the only one, back then) structural property of the Web: the graph that has as nodes web pages and as edges the hyperlinks between them.
In the meantime, a lot of other structural properties that govern subsets of the Web have emerged. To speak of what is today one of the most popular services, Facebook has its own very powerful structure-defining information: the connections between people, that can be translated in a huge social graph. Although few people view it this way, the popular “People You May Know” tool is actually a search engine, built on top of this structure. Instead of being a text, the input consists in your own connections, and instead of receiving web pages as results, you receive recommendations of people that you may know. Although probably the more appropriate term for this tool would be “recommendation engine”, in a more broader sense it is still a search engine: it receives an input, and, based on a structure, it returns relevant information.
Trying to compete with Google at its own game is a loosing anachronistic strategy. Thinking that is possible to build a better service, that uses a similar approach and pretty much the same structural information used by the one that has been constantly adapted and tuned for the last decade by some of the brightest software engineers the world has ever seen is just silly. Still, there are people that are trying to do exactly this, and that I think is proof of lack of vision and perspective.
The future of search lies in taking advantage of the emerging patterns that form new structures on the Web in a useful way. These may be browsing patterns, social structures, large sets of formatted data, or any kind of information that has laws that govern it and make it possible to algorithmically translate it into a searchable structure.
TasteKid is my attempt of taking advantage of these new structures that have emerged on the Web. There are a lot of things to be done in order to improve this service, and its future is still fragile in the face of such an unpredictable industry. But I am excited to work on a project that explores the new possibilities the Web has to offer and to witness the positive feedback it receives from people who actually put this search engine to use for something that, at least for now, few other services offer. Thinking outside the box and understanding that the results of search engines don’t always have to be web pages that contain the searched text is the key of unlocking the potential that the modern Web has to offer to developers today.
