NEXT: A system to easily connect crowdsourcing and adaptive data collection

Obtaining useful crowdsourcing results often requires more responses than can be easily collected. Reducing the number of responses required can be done by adapting to previous responses with “adaptive” sampling algorithms, but these algorithms present a fundamental challenge when paired with crowdsourcing.

At UW–Madison, we have built a powerful crowdsourcing data collection tool called NEXT (http://nextml.org) that can be used with arbitrary adaptive algorithms. Each week, our system is used by The New Yorker to run their Cartoon Caption contest (http://www.newyorker.com/cartoons/vote). In this paper, we will explain what NEXT is and it’s applications, architecture and experimentalist use.