Reddit and the Marathon Bombers: The Wise Way to Crowdsource a Manhunt : The New Yorker

That proposition may be true. But Reddit’s failure isn’t evidence for it. To begin with, it’s a bit facile to frame this story as a competition between “the crowd” and “the experts,” since the official investigation wasn’t relying on a couple of experts, but rather had its own crowd at work, one made up, in Bilton’s words, of “thousands of local and federal officials.” More important is that the Redditors faced a simple, but insuperable, obstacle when it came to identifying the Tsarnaevs, namely that the two brothers were not, as far as I can tell, in any of the photographs that were widely available before Thursday morning. The footage that convinced investigators that the Tsarnaevs were prime suspects was the footage from the Lord & Taylor surveillance cameras, which hadn’t (and still hasn’t) been released to the public. This is an obvious point, but one that’s been overlooked: Reddit had no real chance of identifying the right suspects because it didn’t have access to the information that mattered. (Had the clip of the Tsarnaevs walking down Boylston Street been publicly available last Tuesday, I don’t think there’s any doubt Redditors would have flagged them as suspicious.) Whatever the value of the wisdom of crowds, it isn’t magic: you can’t ask the crowd to find someone that, in a sense, it’s never seen.

You can certainly fault the Redditors for not recognizing the limits of their own knowledge and for jumping to conclusions (even if a good deal of that jumping was done by the national press). But this isn’t a failing that’s specific to Reddit—on the contrary, official investigations fall prey to it all the time. Richard Jewell, of course, was seized on almost immediately by authorities as a prime suspect in the 1996 Olympics bombing, and continued to be treated as such for months before being cleared. The government investigated and harassed Steven Hatfill for years in connection with the 2001 anthrax mailings, before finally backing down. And the F.B.I., on the basis of faulty fingerprint analysis, accused Brandon Mayfield, a Portland lawyer, of assisting with the train bombings in Madrid in 2004; arrested him; and refused to acknowledge its mistake for weeks after Spanish authorities had definitively cleared him. These misidentifications were far more damaging and longer-lasting than anything Reddit did last week, yet one would hardly take them as per se evidence that the F.B.I. should stop investigating crimes.

Of course, saying that official investigators also mess up is hardly a ringing endorsement of crowdsourced sleuthing. And the truth is that if Reddit is actually interested in using the power of its crowd to help the authorities, it needs to dramatically rethink its approach, because the process it used to try to find the bombers wasn’t actually tapping the wisdom of crowds at all—at least not as I would define that wisdom. For a crowd to be smart, the people in it need to be not only diverse in their perspectives but also, relatively speaking, independent of each other. In other words, you need people to be thinking for themselves, rather than following the lead of those around them.

On Reddit, though, it was hard for people to evaluate the photographic evidence independently—the images that were posted had lots of comments attached to them (and were often marked up with circles and arrows)—and the way Reddit’s comment threads work, with certain comments being promoted and featured more prominently than others, meant that some commenters exerted an inordinate influence over the group as a whole. There was more dissent and disagreement in the comment thread than you’d think. But, on the whole, instead of offering up lots of diverse, independent opinions, there was a tendency for Redditors to herd together, taking their cues from the conclusions that others before them had reached. (This was especially true of the information cascade that led to the witch hunt for Sunil Tripathi, as Alexis Madrigal has written.) It was a process that ended up fostering groupthink more than collective intelligence.

It doesn’t have to work this way, though. For example, in a situation similar to the marathon bombings, Reddit could simply post the available images without comment (save, perhaps, for time stamps and where the pictures were taken), and allow users to sift through them, looking for both incriminatory and exculpatory evidence (like, say, people carrying black backpacks near the bomb sites, or still carrying them once the bombs had already gone off), while creating a voting system that would let people independently vote on which images they thought were worth further investigation. Once a critical mass of users had perused the images, Reddit could aggregate the results, and pass them on to the authorities. (The results of the voting wouldn’t even have to be made public, if people were concerned about false accusations.) This would take advantage of the crowd’s ability to filter through lots of images quickly, while also preserving the diversity and independence that make crowds smart. The point of this wouldn’t be to compete with, let alone replace, the work of official investigators, but rather to provide, as it were, another set of eyes, one that might see something that investigators missed.

If Reddit were looking for a model to follow, it could use NASA’s Clickworkers experiment, which in 2000-01 let tens of thousands of amateurs look at photos of Mars in order to identify craters on the planet and classify them by age. That study found that the aggregated judgments of the amateur “clickworkers” were “virtually indistinguishable from the inputs of a geologist with years of experience.”

The problem from Reddit’s perspective, of course, is that this method of sleuthing would be far less exciting for users, and would probably generate less traffic, than its current free-for-all approach. The point of the “find-the-bombers” subthread, after all, wasn’t just to find the bombers—it was also to connect and talk with others, and to feel like you were part of a virtual community. But valuable as that experience may have been for users, it also diminished the chances of the community coming up with useful information. Reddit has done an excellent job of being engaging. Now it needs to figure out if it wants to be effective.

Photograph: F.B.I./A.P.