May 17-18, 2018
Park Inn by Radisson Pulkovskaya
Pobedy Square 1, St. Petersburg

Testing with a crowd

Day 1 /  / Зал 1  /  RU / Хардкор. Сложный низкоуровневый доклад, требующий от слушателя знаний технологии.

Crowdtesting is one of effective ways to scale up manual testing and allow in-house testers to focus on more complicated job. With hundreds of people available for the testrun 24/7, it allows to significantly increase the speed of testing and cover a large diversity of environments.

We will speak about our experience in setting up crowdtesting in Yandex and highlight major steps of this process: how to select and train performers; how to control quality; how to prepare the infrastructure for testing with a crowd; how to combine crowd force with highly qualified in-house QA specialists.

Download presentation
Olga Megorskaya
Olga Megorskaya

Head of crowdsourcing platform Yandex.Toloka and department of human intelligence tasks at Yandex.

Currently is in charge of scaling up and applying crowdsourcing techniques to production processes which require human labour, such as producing the training data for AI, call centers, service support teams, translation, and many others. One of the latest projects was to set up crowdsourcing practices for manual testing at Yandex.

Sphere of research interest — mathematical background of crowdsourcing and subjective human evaluation; author of papers on this subject.