Posts Tagged ‘online search’

Learning Through Online Search: Some Thoughts

March 1, 2009

I searched for the phrase “calories per dollar” after having calculated the amount I spent on a recent grocery purchase. At the end of the second page of results, I found a link to the blog Marginal Revolution and was intrigued about what this left-leaning blog might have to say about the “calorie per dollar” issue. On the post, I found a link to an article in the New York Times by the celebrated food journalist, Michael Pollan, which turned out to be an instructive essay on the effect of national food subsidy policy on the prices of food items. (His essay makes his point that the structure of the “food bill” — the crops it supports and doesn’t support — greatly affects the distribution of prices along the calorie/dollar curve.)

This is a good example of the limitations of search when it comes to researching a topic, in a way that we learn about the greater forces that affect something we care about. When I started out reading about the topic of caloric prices, I didn’t think that an important part of the puzzle was the “food bill.” And Pollan’s article wasn’t even a search result — how could it be, when the phrase “calories per dollar” was not mentioned in it, even though it was a common phrase in other articles?

What this illustrates is the problem in the world of online search of what I call, “intent optimization.” Intent optimization is the ability to insert into a piece of content all the phrases that represent the interests of the people who might benefit from that piece of content. People searching for information about the price of a calorie need to read Michael Pollan’s article but that’s not something that online search can surface.

In order to do so, both content and search terms have to be mapped to a universe of concepts that they relate to, so that they can then be related to each other via this mapping. Because this information is key to the competitive advantage of a search company, it will not be released in any significant format to the public.

However, this information is going to become key to the public good, because we will all benefit from having a better understanding of the zeitgeist, in order to find communities of interest that can support our needs and to find the right pieces of information relevant to our personal development. Therefore, it behooves the open source community to create a service that gathers the search stream and the click-through data and makes it available in the public domain. In effect, it’s an anarchic movement to reclaim the knowledge of the community from the control of corporate interests.

There is no conflict here with the interest of profit-making because the knowledge in the public domain doesn’t allow in itself the creation of an auction market for advertising, which is what an online search engine like Google makes its revenues from. This service will simply aggregate searches and list trends among the most common ones, as well as relationships between different search terms. In the example above about “dollars per calorie,” this service might have detected a significant relationship between that term and, say, “food bill” or “Michael Pollan.”

The service will in fact simply re-purpose results from existing online search engines, and hence all click throughs (that is, search traffic) and ad clicks (that is, paid search revenue) will flow only to the underlying search engines providing the results. Moreover, as the service will retain minimal session information in order to protect individual privacy, it will lack a critical piece of information that forms the competitive advantage for search engine businesses.

Advertisements