Today, in their infinite wisdom and right before the weekend so they could deal with all complaints on Monday, the Mechanical Turk team decided to roll out a fundamental changes to the Requester Web UI.
Specifically, they decided that choosing which workers you want to work on your HITs is now an "Advanced" option, much like choosing paper or plastic at the grocery store is an advanced option or deciding whether you want to buy brand name or generic medicine is for advanced users only. Of course, the things that are really advanced are woefully poorly described and left out of the Requester UI, but we all knew that already.
This correlated with a big change on the Worker's site: the Categorization Masters and Photo Moderation Masters were combined into one big group, which, much to the surprise of everyone, led to 20,686 "Masters"!
So, how does this all play into the requester UI? Amazon, the sleeping giant, decided that instead of policing their site and helping requesters get better quality work by making it easier to create custom qualifications, they just wanted more money. A noble goal (by some definitions of noble), and to achieve it, they made Masters the default option for all new projects, and hid how to qualify workers:
Doesn't it just feel wrong to select "Customize Worker Requirements"? You're basically being told your results will be poor quality. I've been looking at the mTurk worker site, and I've noticed that some poor university students who run HITs have been oblivious to the change, and are likely about to run over their budgets and thrown out of school, all the while wondering why their results were so slow to be delivered:
So, be aware that this has changed when posting new HITs with the web UI. If you use the mTurk API or CLT, this doesn't affect you.
As to how workers are taking it, I have a 99.9% approval rating and 22,000 approved HITs and am seriously looking into finding another way to make ends meet. There's nothing like not being able to get a license that not even the issuing body will tell you the requirements for, and quite frankly I'm quickly becoming sick of it. This sleight of hand with the Requester UI is a dirty trick that will confuse requesters and do little else.
Even if you win the rat race, you're still a rat. ~ William Sloane Coffin
Until next time!
Excellent post! I find it very interesting that Amazon apparently thinks it's okay to lie about the number of workers who have the Masters qualification. A popular batch requester made their HITs Masters only today(Kennedy/Edwards). I e-mailed them and explained about custom qualifications but I was blown off.
ReplyDeleteThey replied with "Thank you for your email and interest in our HITS. We have decided to only allow Master Qualifications due to the abuse of our process."
I've already began spending more time on other sites that allow me to earn money. Maybe one day Amazon will pull their heads out of their asses but I doubt it.
I predict that regular workers(Non-Masters) will slowly be driven from the MTurk workforce. Short of a lawsuit, Amazon is always going to care more about lining their pockets with cash than they care about their customers.
Hello Cambria,
DeleteI wrote an interesting post about Edwards yesterday. You might like it. :)
Aah - Amazon, trying to be the "Master" of deception? The so-called "Master's" qualification is nothing more than a way for Amazon to profit even more from people who post jobs, and people who perform the jobs.
ReplyDeleteAmazing to me that people who post jobs on Amazon's "Mechanical Turk" board are buying into Amazon's sales pitch about "some" people being "Masters" of some sort.
They fall for Amazon's sales pitch and pay 3 times what they would otherwise.
The majority of people who pursue work on the Amazon board find themselves excluded from the default "Master" qualification required jobs, and the people who post work pay Amazon twice as much as they pay the people who perform the work for them.
Amazon takes 2/3 of the cost of each "Master" qualified job.
Amazon doesn't do anything to "deserve" 2/3 of the cost of a job. That money could, and should go to the people who perform the work.
Unethical behavior on the part of Amazon? Definitely!
I'd love to be able to adjust the worker requirements to not require Masters for my jobs. Where can I find this "Advanced" menu?
ReplyDeleteYou can find instructions here: http://turkernation.com/showthread.php?14830-mTurk-WebUI-defaults-to-using-Masters-costs-20-extra-how-to-turn-it-OFF
Delete