Markiter and Mechanical Turk
Markiter currently uses Amazon's Mechanical Turk as the primary method for engaging voters for testing campaigns. Mechanical Turk is a web service created by Amazon.com based on the belief that there are certain things that humans can do better and faster than a computer. Image recognition is a perfect example; a human can easily tell what products, or items are in a picture much more accurately than a computer can. This is why Amazon's official IPhone application currently uses Mechanical Turk for one of it's key features. An IPhone user can take a picture of an item, send it to Amazon, and Mechanical Turk workers will identify the item, and return the product detail page on Amazon that best resembles the item. Wired magazine has some more information about it here.There have been numerous studies done to validate our beliefs that Mechanical Turk is the perfect venue to leverage the wisdom of crowds for market research purposes. The main concerns we had before selecting Mechanical Turk on behalf of our customers were the distribution of voters, and the quality and accuracy of voters. Let's take a look at the independent research we were able to uncover for each of these.Voter Distribution:
A study was recently done to study the distribution and demographics of Mechanical Turk voters. Here is a summary of their findings. You can read the full study and see more graphical breakdowns here.
- 75% of voters are from the States, and not a third world country
- Almost 50/50 distribution between male/female participants
- Large majority of voters were between 21-40.
- Over 50% of Mechanical Turk voters have a Bachelor's degree
- The income distribution of voters mirrors the income distribution of the US
Take a look at the some real Mechanical Turk workers, and the experiment to find out more about them:
<img src="http://www.markiter.com/media/images/faces_of_mechanical_turk_small.jpg">
Enrique Andres Ferraro from the University of Liverpool did an extensive study into the characteristics of Mechanical Turk, and concluded the following:
"We conducted an analysis of worker’s demographic characteristics with a sample of 1428 workers, which we contrasted and compared with the US Census of 2000. Our study reveals that The Amazon Mechanical Turk attracts workers from all segments of a population, largely in proportions matching the population from which they were drawn, and that the system represents a portal through which market research can be conducted easily and cost-effectively without incurring some of the sacrifices in validity inherent in captive panels or indirect online research."
You can see his full dissertation here
Voter Quality:One fundamental question about Turk that raises concern is this: "How do you control the quality of the responses?
Aaron Koblin and Takashi Kawashima have answered this question. Exhibit A: Their Ten Thousand Cents project
<img src="http://www.markiter.com/media/images/tenthousandcentsbill.jpg">
They broke a $100 bill into 100,000 segments and asked Mechanical Turk workers to draw their piece of the bill. With no validation on the submitted picture, its clear that a large majority of responders followed through with the requested task. While it is true that a few people that did not do their part, this experiment goes to show the normalization power of crowds. Despite the few exceptions, the sum of the overall community will point you down the right path. Applying this community to market research provides a powerful value proposition.
This is no different from traditional market research. There are bound to be a few people that don't take the time to answer questions correctly, or randomly fill out surveys, but market research is still extremely valuable because removing the outliers gives you a clear picture on the general populations views and opinions.
