Assignments

What do I do?

There will be a set of assignments that will be announced as the course progresses, in which you'll:

• Become a crowd worker.
• Survey & Analyze existing crowdsourcing & social computing systems.
• Replicate results from an academic paper.

Why do this?

Hand-on exercises and implementations are a fun and effective way to learn.

How do I submit?

We'll create an assignment in KLMS for each assignment.

Late Policy

You'll lose 10% for each late day. Submissions will be accepted until three days after the deadline. After then you'll get 0 on that assignment.

Assignment 1: Be a crowd worker

Due: Before class on 9/7 (Thu)

What do I do?

In this assignment, you'll experience what it's like to be a crowd worker. In class, we walked through the MTurk user interface, both as a worker and as a requester, and discussed its strengths and limitations. Now it's your turn to pick a crowdsourcing platform of your choice (sorry, MTurk doesn't count!) and be a worker yourself. We'll focus on voluntary crowdsourcing platforms, which are quite different from platforms offering monetary reward in many ways.

Pick one of the following platforms:

• Lab in the Wild
• Galaxy Zoo (or any of the Zooniverse projects)
• fold.it
• Eyewire

You need to spend at least one hour as a crowd worker using the platform. Show us some visual proof of what you've done with screenshots (e.g., example task, how much work you've done, your score in a game, etc.).

Your report

In your report, please reflect on your experience by answering the following questions:

1. What is the name of the platform you picked?
2. Why is this crowdsourcing? What data, service, or idea is it crowdsourcing?
3. Walk through your task finding, performing, and learning experience with screenshots.
4. What did you like about the platform? Why do people come to this platform?
5. What problems did you run into? How would you improve the platform and the tasks to attract more volunteers?
6. In the scale of 1 (not really) - 10 (very much), answer the following: "I think I contributed meaningful work that helps the goal of the platform."
7. In the scale of 1 (not really) - 10 (very much), answer the following: "I personally enjoyed working on the tasks."

How do I submit?

KLMS

Assignment 2: Analyze Crowdsourcing Platforms

Due: 11:59pm on 11/17 (Fri)

What do I do?

In this assignment, you'll analyze THREE existing crowdsourcing platforms, using the dimensions we collectively identified by learnersourcing.

Here are some notes:

1. Examples that have been previously discussed in class cannot be used.
2. Platforms you pick should be different from one another in a significant way (e.g., you cannot pick three Zooniverse projects).
3. You can define crowdsourcing broadly, as long as you can explain why (e.g., Wikipedia? Uber? Crowdfunding? Recommender systems?).
4. To help you find good examples, Wikipedia has a crowdsourced collection of crowdsourcing projects. But you should also explore on your own!

Dimensions

We'll use the following seven dimensions, which were top-rated in our own crowdsourcing effort to rank the dimensions that matter the most in analyzing a crowdsourcing system.

• Quality control - how to ensure valid results?
• Motivation - why would a crowd worker do this?
• Task design - make it as easy (or even enjoyable) as possible for the worker to do the task
• Aggregation - how are results from multiple workers combined?
• Result - is there any data or idea resulting from the process?
• Accessibility - how can potential workers find the fittest task?
• Process order - in what order is the work processed between computer, worker, and requester?

We understand that it might be difficult to precisely extract some of the dimensions as crowd workers of a platform. The underlying mechanism might be hidden from the crowd. In such a case, please (1) indicate that the information is not publicly available, but (2) write your thought on what might be happening under the hood.

Your report

In your report, please analyze each platform by answering the following questions:

1. What is the name of the platform? URL?
2. Why is this crowdsourcing? What is it crowdsourcing?
3. How did you find out about the platform?
4. Analyze the platform using the seven dimensions above.
5. Discuss one strength and one weakness of the platform.
6. Include a screenshot.

How do I submit?

Create a PDF version of your report, and submit on KLMS