search

facebookresearch / ParlAI

A framework for training and evaluating AI models on a variety of openly available dialogue datasets.
https://parl.ai
MIT License
10.45k stars 2.1k forks source link

Add 'taskntalk' dataset of synthetic shapes to ParlAI. #446

Closed kdexd closed 6 years ago

kdexd commented 6 years ago

Task N' Talk is a goal-based cooperative conversation game between two agents - a questioner and answerer. It was described in Kottur et al. '17 "Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialog" (arxiv).

I implemented this paper using ParlAI (lang-emerge-parlai). As it works fine, this issue is created to group a bunch of incoming PRs related to the task.

kdexd commented 6 years ago

Next up: Our QBot and ABot in parlai/agents

jaseweston commented 6 years ago

Cool.. are they general agents that might work on some of the other parlai tasks? ( https://github.com/facebookresearch/ParlAI/blob/master/docs/source/_static/img/tasks2.png?raw=true )

If yes, awesome! If no, they should go in a projects/ folder instead

On Mon, Dec 11, 2017 at 11:26 PM, Karan Desai notifications@github.com wrote:

Next up: Our QBot and ABot in parlai/agents

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/facebookresearch/ParlAI/issues/446#issuecomment-350941703, or mute the thread https://github.com/notifications/unsubscribe-auth/AKjk-LCqV2yguUOVUGH4gqs3K5Jkh5dWks5s_gB4gaJpZM4Q-Xui .

kdexd commented 6 years ago

I think they should be re-usable for the VisDial task. If I avoid hardcoding the dimensions of internal modules. I am speaking in context of this figure:

image

Taken from Das et al. 2017a Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning. Even this paper conducts experiments on both, Task N' Talk and VisDial.

Of course, the architecture hyperparameters are different but it has a similar structure. I will try to generalize well enough so that my agents can be used for both tasks. :)

kdexd commented 6 years ago

For completeness, this figure is from Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialog. image

ListenNet works like Encoder and SpeakNet works like Decoder.

kdexd commented 6 years ago

@alexholdenmiller @jaseweston Hi, I created the agent classes and made a PR. Now in the parlai/projects, I can subclass the dictionary agent and provide task specific mappings (for example, X - 1). Also I shall subclass my agents and simply override dictionary_agent() staticmethod. This should serve as an example to show easy subclassing of these agents as per requirement.

kdexd commented 6 years ago

Next up: A training script in projects and a tutorial doc on how to use these agents or extend them as per necessity.

jaseweston commented 6 years ago

what is the status here?

kdexd commented 6 years ago

I got engaged in academics lately. It is almost done - the teacher is not reading from validation data right now, just making a PR for it in a few days!

jaseweston commented 6 years ago

ok thanks!

On Sun, Jan 28, 2018 at 11:03 PM, Karan Desai notifications@github.com wrote:

I got engaged in academics lately. It is almost done - the teacher is not reading from validation data right now, just making a PR for it in a few days!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/facebookresearch/ParlAI/issues/446#issuecomment-361134803, or mute the thread https://github.com/notifications/unsubscribe-auth/AKjk-K3iW98hhiwRKb5RWYHBzJioFLozks5tPUMSgaJpZM4Q-Xui .

jaseweston commented 6 years ago

should we keep this open?