facebookresearch / EmpatheticDialogues

Dialogue model that produces empathetic responses when trained on the EmpatheticDialogues dataset.
Other
450 stars 63 forks source link

The Emotion Classifier #10

Closed yuboxie closed 5 years ago

yuboxie commented 5 years ago

Hi there,

I have a few questions regarding the emotion classifier in the EmoPrepend model:

  1. Is the pre-trained classifier available?
  2. What data did you use to train the emotion classifier? Is it the context-prompt pairs in the ED dataset?
  3. It is mentioned in the paper that history utterances are concatenated into one and sent to the encoder, so in the case of EmoPrepend, is it correct that the input has the following format: [emotion_label_1] utterance_1 [emotion_label_2] utterance_2 ...?

Thanks for your response!

EricMichaelSmith commented 5 years ago

Hi there! To answer your questions:

  1. No, unfortunately this classifier is not available at this time.
  2. From the paper: we trained the classifier to predict the emotion label from the description of the situation written by the Speaker before the dialogue for the training set dialogues of ED. This description of the situation is separate from the Speaker/Listener utterances in the ED conversations themselves, but they're still available in our released dataset. (@hadasah - feel free to chime in if any of this is incorrect!)
  3. No, the emotion label is only added once, at the very beginning of all of the utterances in the history. The code for that is at https://github.com/facebookresearch/EmpatheticDialogues/blob/master/empchat/datasets/empchat.py#L128
yuboxie commented 5 years ago

Thank you for your response! Actually the second question was, on what data did you train the emotion classifier that predicted the emotion labels in ED? Thanks!

EricMichaelSmith commented 5 years ago

Sure thing! For the emotion classifier that predicted the emotion labels used for EmoPrepend-1/3/5, the data that we trained on was the ED dataset itself, namely, the situation/emotion-label pairs. Lemme know if this answers your question - otherwise I'm happy to elaborate further!

yuboxie commented 5 years ago

If you trained the emotion classifier on the ED dataset, why did you do the prediction on ED again in the code (empchat.py)? Also, it seems that you prepend the topic instead of the emotion label: https://github.com/facebookresearch/EmpatheticDialogues/blob/f6352667bb1547ebeac68cd07932597a49f0167d/empchat/datasets/empchat.py#L135

EricMichaelSmith commented 5 years ago

Doing the prediction on the ED dataset allows us to prepend more than 1 emotion label: for instance, we can prepend the top-5 most likely emotions, as seen with EmoPrepend-5 in table 7. And, yes, for TopicPrepend we prepend one or more topic labels using the mapping in topicmap: if we're doing EmoPrepend, though, our emotion label won't be in topicmap, and so the prepended label will just be the emotion itself (see the second arg in topicmap.get()).

yuboxie commented 5 years ago

Ah I see :) Thanks for the clarification!

EricMichaelSmith commented 5 years ago

No problem :)