Amazon Mechanical Turk: The "Artificial Artificial Intelligence" Marketplace
Inside was a small, cramped chamber. A worn leather cushion. A single candle stub. A half-eaten loaf of bread. And a tarnished silver mirror, angled upward so that its occupant could see the chessboard through the Turk’s transparent chest piece. Paul touched the mirror. It was still warm.
Overall, Mechanical Turk is a powerful platform for crowdsourcing tasks that require human intelligence. While it offers many benefits, it also presents some challenges and limitations that need to be addressed.
: If one Worker consistently provides answers that differ from the majority, the system would automatically "flag" them for manual review or prevent them from taking future HITs in that batch.
He heard a footstep behind him. Johann stood in the doorway, his face tired, his eyes sad but not angry. He said nothing. He simply knelt beside Paul, pointed to the mirror, then to the chessboard, then placed a finger over his own lips.
Requesters define the task, set the payment, and specify worker requirements. Once a worker completes a HIT, the Requester reviews the work and, if satisfied, approves the payment, which is then facilitated by Amazon.
MTurk offers several benefits, including: