facebookresearch / ParlAI

A framework for training and evaluating AI models on a variety of openly available dialogue datasets.
https://parl.ai
MIT License
10.49k stars 2.1k forks source link

How can I create a custom MTurk task that connects two turkers? #1929

Closed josharnoldjosh closed 5 years ago

josharnoldjosh commented 5 years ago

Hello, I am trying to create a custom task very similar to personachat or multi agent chat with HTML where two turkers are matched and chat between each other.

Currently, I am working on onboarding the turkers where two turkers are matched.

I created a run.py where I include an onboarding world:

def run_onboard(worker):
    world = GanDrawOnboardWorld(opt, worker)
    while not world.episode_done():
        world.parley()
    world.shutdown()

And then I created a simple world in worlds.py

class GanDrawOnboardWorld(MTurkOnboardWorld):
    def __init__(self, opt, mturk_agent):                
        super().__init__(opt, mturk_agent)

    def parley(self):        
        self.mturk_agent.observe({'id': 'System', 'text': config["onboard_message"]})        
        act = self.mturk_agent.act()
        self.mturk_agent.observe({'id': 'System', 'text': config["waiting_message"]})
        self.episodeDone = True

Then I created an onboarding_index.html page that's really simple:

{% extends "core.html" %}

{% block main_pane %}
<div>
    <p id="display"></p>
</div>
{% endblock %}

{% block additional_scripts %}
<script type="text/javascript">
    function handle_new_message(new_message_id, message) {
        var message_text = message.text;
        document.getElementById('display').innerHTML = message_text;
    }
</script>
{% endblock %}

I figured out by extending the core.html I can implement the handle_new_message function to receive packets via websocket from the agent.observe() function.

However, I am not sure where to go next to ensure I can connect two turkers to a task and successfully complete the onboarding world episode and move to the next.

I've checked the documentation and it is very high level. The example code looks high level too.

How can I implement the functionality for two turkers to be matched, then as soon as they are matched, the onboarding episode finishes and they begin the task? I.e, move from the onboard task to the "start task":

        mturk_manager.set_onboard_function(onboard_function=run_onboard)           
        mturk_manager.start_task(
            eligibility_function=check_worker_eligibility,
            assign_role_function=assign_worker_roles,
            task_function=run_conversation,
        )

Thanks!

JackUrb commented 5 years ago

Hi @josharnoldjosh - this level of functionality is abstracted away such that you don't need to think about it. If the onboarding world returns episode done as true, the workers are put into a waiting pool until another eligible worker also is in the pool, and then the pairing process happens automatically.

I would suggest you move over to the react implementation as shown in the react demos, as the html implementation is being deprecated in the near future (less stability, more bugs, harder to maintain).