Compare Prime Panels to Mechanical Turk

Mechanical Turk

Amazon Mechanical Turk is the premier participant recruitment platform for Web-based social and behavioral research studies. However, MTurk has numerous limitations including a very small participant pool, overused workforce, almost exclusive use of convenience samples, and lack of anonymity.

Mechanical Turk Alternatives

Before Mechanical Turk became popular, hundreds of participant recruitment platforms had been extensively utilized in public opinion and market research. Public opinion and market research platforms offer significant advantages including a participant pool of millions of individuals, access to a worldwide population, and the ability to target hard-to-reach participants.

Prime Panels

TurkPrime is the only platform that offers access to participants both on and off Mechanical Turk. The TurkPrime Prime Panels service will help you to conduct studies outside of Mechanical Turk, while maintaining high levels of data quality.

Deciding on Where to Recruit Participants

Like any important research decision, deciding on which platform to use for participant recruitment requires careful consideration. Each platform offers its own benefits and may have specific limitations. Below we provide an overview of the relative strengths of Mechanical Turk and TurkPrime Panels. The overview is divided along four domains: Population, Design, Data Quality and Quality Controls.

Population
Mechanical Turk Prime Panels
Size < 20,000 10 million
Targeted recruitment Difficult On MTurk
More options with MTurk Panel Service
Easy
Hard to reach samples Very difficult/impossible Medium difficulty
Global Limited to US, Canada, India Almost any country
Verified identity No Over 1 million registered voters
Representativeness Highly skewed More representative
Design
Mechanical Turk Prime Panels
Launch study using URL link from Qualtrics, Survey Monkey or any other Yes Yes
Longitudinal studies Easy with TurkPrime Toolkit In development
Communication Easy with TurkPrime Toolkit In development
Payment Direct Control In development
Bonus Easy with TurkPrime Toolkit In development
Approval/Rejection capability Yes Yes
Quality
Mechanical Turk Prime Panels
Replication Extensively shown to replicate classic finding from personality, cognitive, and social psychology Extensively shown to replicate classic findings from personality, cognitive, and social psychology
Survey studies Quality level high Quality level high
Experimental manipulations Quality level high Quality level high
Open ended responses Quality level can be high Currently being tested
Dropout from longitudinal studies 20% to 50% Currently being tested
Tasks requiring creativity and very high attention levels Quality level can be high, bonuses may improve quality High, but only at higher prices. Bonuses may improve quality
Quality Control
Mechanical Turk Prime Panels
High quality participants Only those who completed at least 100 studies and have a high Approval Rating Everyone who passes a qualifying study consisting of a series of attentional manipulation questions, captcha, and basic language requirements.(You may use your own)
Prior participation in many psychological studies Very high Very Low
Familiarity with psychological manipulations and procedures Very high Very Low
Effect sizes for common manipulations Lower Higher