#1 – Implement Systematic Job Retrieval from Mechanical Turk

[ansaqib]

Context

HUMAN Protocol efficiently handles data annotation tasks by distributing them to its global workforce, ensuring their accurate completion, and managing worker payments. With a significant pool of workers ready for data annotations, this issue aims to leverage Mechanical Turk for the source of jobs.

Problem

We need a way to efficiently locate, retrieve, and complete tasks from Mechanical Turk to maximise our workforce capacity and potentially offer faster and cheaper services.

Solution

1. Develop a daemon that interfaces with AWS to continuously scan for available jobs on Mechanical Turk.
2. Convert suitable jobs into a format compatible with the HUMAN Protocol.
3. Once the job is completed, fetch the results and post them back to Mechanical Turk.

Acceptance Criteria:

• The solution should be able to get available jobs from Mechanical Turk
• Should convert Mechanical Turk jobs into a format compatible with HUMAN protocol without data loss.
• After task completion at HUMAN protocol, results should be posted back to Mechanical Turk in the required format.
• Jobs should only be taken when deemed to be profitable.
• Ensure seamless operation with minimal manual intervention.

Technical Details:

1. Daemon Creation:
   • Utilise AWS SDK to interface with Mechanical Turk.
   • Schedule periodic scans for new tasks.
2. Job Conversion:
   • Identify the structure and data format of basic data annotation jobs on Mechanical Turk.
   • Develop a conversion tool or middleware to transform these jobs into our protocols format.
3. Result Posting:
   • Once a task is completed on our protocol, format the result to meet Mechanical Turk's requirements.
   • Use AWS SDK to post the results back to the respective task on Mechanical Turk.

Guides as to how to start/direction:

• Begin with the daemon creation. Ensure that it can authenticate and read available tasks on Mechanical Turk.
• Focus on the job conversion mechanism. It might be beneficial to analyse a few diverse job types on Mechanical Turk to ensure our conversion tool handles a broad spectrum of tasks.
• The primary job types we will be looking to capture and complete are:
  • Image Categorisation
  • Image Label Multiple Choice
  • Image Label Area Select
  • Text Multiple Choice
  • Open Query
  • Image Label Semantic Segmentation
  • Image Label Area Adjust
  • Text Free Entry
• Lastly, concentrate on posting the results back. Make sure the data is correctly formatted and no errors occur while posting back to Mechanical Turk.

Bounty Reward:

• The successful completion of this task will be rewarded with a bounty of 3000 USD in HMT.

Additional notes:

• Ensure all code is properly documented to ease future maintenance and further development.
• The solution should come with a set of unit and integration tests to verify functionality.
• For any questions or clarifications regarding this task, please reach out to a member of the HUMAN tech team on the #tech channel in Discord.
• Preferred languages: Python, JavaScript

[zain581]

I am reaching out to express my interest in working on the initiative to integrate Mechanical Turk with the HUMAN Protocol. I find the project intriguing, I am eager to contribute to the development of the proposed solution. Using the AWS SDK to interface with Mechanical Turk and scheduling periodic scans for new tasks will be the initial steps. Looking forward to hear .

[ansaqib]

Sure please pick a time https://calendly.com/saqib-hmt

[zain581]

@ansaqib I haven't heard from you recently.let me know if you need anything else from my side .