Occassional thoughts about orienteering
|
|
| okansas.blogspot.com Occassional thoughts about orienteering |
|
|
Saturday, July 10, 2004 Results of the experimentThanks to everyone who sent an email to help me with my experiment. I really appreciate it.Now, before getting on to the results, let me give you a bit of background. "The Wisdom of Crowds" I recently read a book called "The Wisdom of Crowds" by James Surowiecki. The book mixes ideas from economics, sociology, psychology and probably some other disciplines to look at the idea that groups (i.e. "crowds") can sometimes be very good at solving problems. Without getting in to all the details, Surowiecki shows that under certain conditions crowds can solve certain problems. As I read the book, it struck me that making route choice decisions fit the type of problems that crowds do a good job of solving. So I came up with an experiment to see how it'd work. The experiment If you're reading this, you probably know what the experiment was. But, just to recap, I'll describe it. I found a route choice problem. I asked readers to tell me which route was fastest and how much faster it was. I planned to compile all the answers, aggregate the information by calculating average time difference between the routes, compare the crowd's performance to the actual race results, and see how the crowd performed. Some more info about the route choice problem The route choice problem I used was published in Skogsport magazine back in 1998. The leg was from the women's course at a World Cup race in Estonia. Hanne Staff ran close to the straight line, skirting the edges of marshes. Karolina Arewang ran out to the trail, climbing a bit but getting some easier running. Staff took 1:46 and Arewang took 2:21. I could just subtract the times and find that the straight route was 35 seconds faster. But, the splits from the rest of the course and the overall results suggest Staff was moving faster the whole time. I adjusted for this and decided that the actual difference in routes was about 27 seconds. What about the crowd? I got about 25 responses. Most, but not all, thought Staff's route was faster. I aggregated the responses in two ways. First, I averaged the time differences for all of the answers. (I entered negative numbers for those who guessed Arewang's route was faster). Second, I averaged only those responses that picked Staff's route, ignoring the responses favoring Arewang's route (about 20 percent of the total responses). By either method, Staff's route is faster. By the first method, Staff's route is faster by 11 seconds and by the second method her route is faster by 16 seconds. The split times suggest a time difference of 27 seconds, and the crowd's average shows a difference of 11-16 seconds. A couple of things strike me as interesting: The crowd got it right. Even though about 20 percent of the individuals (about 5 of 25) picked Arewang's route as faster, the crowd picked the faster route. The crowd's estimate of the time difference looks pretty good to me. You could make an argument that the difference between the crowd's average and the split times is big (11-16 seconds compared to 27 seconds), bit keep in mind that the crowd had very little information about the conditions. Are the woods thick? Is the marsh wet? Was it cold and rainy? Was it hot and humid? What was the contour interval? I don't know the answer to any of those questions. Neither did the crowd. Yet the crowd did a pretty good job of making the best choice and of estimating the difference in time. Why does it matter? Obviously one little experiment doesn't matter. But, what if it turned out that the crowd always got it right? Wouldn't it be cool if you could look at a leg and, without having split times or running the leg, get a good idea of which route was faster and how much faster it was? I think that'd be quite cool. It might even be useful. I can imagine that a course setter could use the crowd to test legs. You could use a crowd to study route choices for future events (imagine being able to begin to understand route choices for the WOC in Japan without having to go to Japan). Thanks again Thanks again to everyone who helped by sending their guesses. posted by Michael | 2:45 PM 
Comments:
Post a Comment
|
|
|
|
|
|
|
 |
||||
|
|||||
