Imagine it is an exciting day at the fat stock and poultry exhibition, somewhere in India, and you wrangle with your mind to estimate the weight of a healthy ox you would like to buy. After some minutes of guessing you finally give up and decide to ask someone. You have two choices – one is to ask a trader who has years of expertise and pick his number. And the other, is a group, say of size 30, of amateur traders, visitors and the experienced and highly skilled trader himself, where you are just going to average their answers. Which one would you choose?
Probably a lousy question to ask someone, as you may think. When I was asked the same thing, I opted for the experienced trader with a ‘duh’ look on my face like most of you since a skilled trader could definitely predict better than a bunch of amateurs. But James Suroweicki’s research says otherwise. Group intelligence work in an amazing manner with its distinctive feature being it is easy to miss and when you catch it, it is difficult to accept.
Let us get out of the boring research hypothesis or findings or whatever they call it for a while and look at something ‘interesting’. You see, research itself suggests that giving expectations to the readers in the beginning would encourage them to go ahead. The risk is the work being declared dull, though pretty interesting it may be, since the expectations given in the beginning would lead to a flawed decision making, according to theories from humanistic evolutionary psychology. Okay no research talk, seriously.
I was on cloud nine when I heard Roger Waters is planning a world tour and hence tried to browse for tickets, in case if they are visiting the city I live in. I have always loved Pink Floyd and ‘We don’t need no Education’ was my daily prayer when I was at school. It is ironic I am pursuing Masters Degree; giving away I can be a hypocrite too. To confirm the news, I typed the keywords ‘Roger Waters’s world tour’ in Google which took a fraction of seconds (with the connection which I had in India three years back, it would have taken five minutes though) publish its results. The first page had ten results, our of 328,000, which were closely related to the Roger Water’s tour- The Wall, 2011-2012. If you are a regular search engine user, these things wouldn’t surprise you. I was, but for a different reason. Suppressing the bliss coming out seeing that they are scheduled to play at the arena nearby and taking away my friends’ fun of witnessing my girly ‘yahooooo’ expression, I made a futile attempt to figure out how a search engine could pick the best choices among the billion web pages entangled in the global network. Sure enough I have heard of the famous ‘Page Rank’ algorithm first designed by Larry Page. He is lucky in a way that naming the algorithm after his made sense, though it is convincing this wasn’t the reason the name came up. Imagine Livermore physicist Dick Post inventing a new ‘ball bearing’ machine and naming it after him. Physics can become interesting sometimes.
Pardon me. Back to the point- The success of the page rank algorithm largely relates to the concept of Group Intelligence (Hence the title G.I. vs Joe, the average Joe. Please ignore if you are bright minded and had already gotten it). The logic behind which the algorithm was designed did not make sense when I was studying computers or perhaps nothing made sense what I studied about computers. Here’s what Google says-
''PageRank capitalized on the uniquely democratic characteristic of the web by using its vast link structure as an organisational tool. In essence, Google interprets a link from page A to page B as a vote, by page A, for page B. Google assesses the page importance by the votes it receives. But Google looks at more than sheer volume of votes, or links; it also analyses the page that casts the vote. Votes cast by pages that are themselves ‘important’ weigh more heavily and help to make other pages ‘important’''.
If it makes sense, then good luck having a future as a computer engineer. Look at Surowiecki’s explanation now. In a fraction of second, what Google does is to ask the entire Web to decide which page contains the most useful information, and the page that gets the most votes goes first on the list. And that page, or the one immediately beneath it, more often than not is in fact the one with the most useful information. To put it in political terms, Google is republic and not a perfect democracy. This is because the final vote is the ‘weighted average’ rather than the simple average like the ox-weighter’s estimate. Nonetheless, the big sites that have more influence over the crowd’s final verdict have that influence only because of all the votes that smaller sites have given them. If the smaller sites were giving the wrong sites too much influence, Google’s search results would not be accurate. In the end, the crowd still rules. To smart at the top, the system has to be smart all the way through.
'Crowd performs better' does not mean if you allow a group of amateurs to run and take their average timing, this will be better than Usain Bolt’s, who wants to play for Manchester United. (I know who he wants to play for is entirely irrelevant, but since it is ManUnited I thought I should write it! It would be pointless though, since he would every time run faster than the ball would travel often missing out).The group intelligence would work only when it comes to decision making provided the group contains at least 30 members satisfying the following conditions- diversity, independence and decentralisation.
Cognitive diversity (and not the cultural diversity) is important in this context. The more diverse people’s thoughts are, the more flexible they could think leading to extreme inputs from them, both positive and negative. When put together through statistical measures, be it simple mean or the weighted average mean, these positive and negative inputs would cancel out by themselves and give us the most appropriate result that we could possibly get. This magic would work only if the individuals are given freedom to think and express which the independence factor is, precisely. And the last thing is- this would best work if they work in a decentralized manner rather than being subjected to a top down decision making process, the very factor that gave the Linux operating system a huge success. Linux is owned by no one and there are no bosses running around instructing people what to do. When a problem arises with the way Linux works, it is solved by its users, who offer solutions. Amazingly this decentralization provides both diversity and independence, making it the most important factor.
With all these said and done, this pretty much sums up why ‘the many’ are smarter than ‘the few’. The impact of this is all around us, though we seem to ignore it, from small things that make up our day to day life to large corporations which create wonders. Taking it would perhaps not make us genius over-night like Page but would certainly enhance our decision making processes and with that being understood, let us be careful in naming our inventions after our's just in case if we did invent one.
Cheers,
G'tam.
Probably a lousy question to ask someone, as you may think. When I was asked the same thing, I opted for the experienced trader with a ‘duh’ look on my face like most of you since a skilled trader could definitely predict better than a bunch of amateurs. But James Suroweicki’s research says otherwise. Group intelligence work in an amazing manner with its distinctive feature being it is easy to miss and when you catch it, it is difficult to accept.
Let us get out of the boring research hypothesis or findings or whatever they call it for a while and look at something ‘interesting’. You see, research itself suggests that giving expectations to the readers in the beginning would encourage them to go ahead. The risk is the work being declared dull, though pretty interesting it may be, since the expectations given in the beginning would lead to a flawed decision making, according to theories from humanistic evolutionary psychology. Okay no research talk, seriously.
I was on cloud nine when I heard Roger Waters is planning a world tour and hence tried to browse for tickets, in case if they are visiting the city I live in. I have always loved Pink Floyd and ‘We don’t need no Education’ was my daily prayer when I was at school. It is ironic I am pursuing Masters Degree; giving away I can be a hypocrite too. To confirm the news, I typed the keywords ‘Roger Waters’s world tour’ in Google which took a fraction of seconds (with the connection which I had in India three years back, it would have taken five minutes though) publish its results. The first page had ten results, our of 328,000, which were closely related to the Roger Water’s tour- The Wall, 2011-2012. If you are a regular search engine user, these things wouldn’t surprise you. I was, but for a different reason. Suppressing the bliss coming out seeing that they are scheduled to play at the arena nearby and taking away my friends’ fun of witnessing my girly ‘yahooooo’ expression, I made a futile attempt to figure out how a search engine could pick the best choices among the billion web pages entangled in the global network. Sure enough I have heard of the famous ‘Page Rank’ algorithm first designed by Larry Page. He is lucky in a way that naming the algorithm after his made sense, though it is convincing this wasn’t the reason the name came up. Imagine Livermore physicist Dick Post inventing a new ‘ball bearing’ machine and naming it after him. Physics can become interesting sometimes.
Pardon me. Back to the point- The success of the page rank algorithm largely relates to the concept of Group Intelligence (Hence the title G.I. vs Joe, the average Joe. Please ignore if you are bright minded and had already gotten it). The logic behind which the algorithm was designed did not make sense when I was studying computers or perhaps nothing made sense what I studied about computers. Here’s what Google says-
''PageRank capitalized on the uniquely democratic characteristic of the web by using its vast link structure as an organisational tool. In essence, Google interprets a link from page A to page B as a vote, by page A, for page B. Google assesses the page importance by the votes it receives. But Google looks at more than sheer volume of votes, or links; it also analyses the page that casts the vote. Votes cast by pages that are themselves ‘important’ weigh more heavily and help to make other pages ‘important’''.
If it makes sense, then good luck having a future as a computer engineer. Look at Surowiecki’s explanation now. In a fraction of second, what Google does is to ask the entire Web to decide which page contains the most useful information, and the page that gets the most votes goes first on the list. And that page, or the one immediately beneath it, more often than not is in fact the one with the most useful information. To put it in political terms, Google is republic and not a perfect democracy. This is because the final vote is the ‘weighted average’ rather than the simple average like the ox-weighter’s estimate. Nonetheless, the big sites that have more influence over the crowd’s final verdict have that influence only because of all the votes that smaller sites have given them. If the smaller sites were giving the wrong sites too much influence, Google’s search results would not be accurate. In the end, the crowd still rules. To smart at the top, the system has to be smart all the way through.
'Crowd performs better' does not mean if you allow a group of amateurs to run and take their average timing, this will be better than Usain Bolt’s, who wants to play for Manchester United. (I know who he wants to play for is entirely irrelevant, but since it is ManUnited I thought I should write it! It would be pointless though, since he would every time run faster than the ball would travel often missing out).The group intelligence would work only when it comes to decision making provided the group contains at least 30 members satisfying the following conditions- diversity, independence and decentralisation.
Cognitive diversity (and not the cultural diversity) is important in this context. The more diverse people’s thoughts are, the more flexible they could think leading to extreme inputs from them, both positive and negative. When put together through statistical measures, be it simple mean or the weighted average mean, these positive and negative inputs would cancel out by themselves and give us the most appropriate result that we could possibly get. This magic would work only if the individuals are given freedom to think and express which the independence factor is, precisely. And the last thing is- this would best work if they work in a decentralized manner rather than being subjected to a top down decision making process, the very factor that gave the Linux operating system a huge success. Linux is owned by no one and there are no bosses running around instructing people what to do. When a problem arises with the way Linux works, it is solved by its users, who offer solutions. Amazingly this decentralization provides both diversity and independence, making it the most important factor.
With all these said and done, this pretty much sums up why ‘the many’ are smarter than ‘the few’. The impact of this is all around us, though we seem to ignore it, from small things that make up our day to day life to large corporations which create wonders. Taking it would perhaps not make us genius over-night like Page but would certainly enhance our decision making processes and with that being understood, let us be careful in naming our inventions after our's just in case if we did invent one.
Cheers,
G'tam.
Excellent and I appreciate it. The originality in the perceptions of your thought reflects well in your article. Good Job! Keep it up!
ReplyDelete