search

behavioral-data / Empathy-Mental-Health

Repository containing codes and dataset access instructions for the EMNLP 2020 paper on empathy in text-based mental health support
154 stars 34 forks source link

How to make predictions? #2

Closed MaLiN2223 closed 3 years ago

MaLiN2223 commented 3 years ago

Thank you very much for publishing the model and the data! I have a question about how to use the trained model. Assume that I already have a .pth model trained and now, I would like to predict the level based on a text input. It seems like the current implementation requires the labels to always be there. Otherwise this lane

loss = lambda_EI * loss_empathy + lambda_RE * loss_rationales does not have loss_empathy and loss_rationales defined.

I figured that logits_empathy should somehow store the data about the empathy but not sure how to interpret those values. [ 0.2010, 1.9501, -2.0850]

Do I understand correctly that argmax would suit here?

Lastly, how should we interpret the result? Say the argmax gives value '2' does it mean that this is the highest level of empathy?

ash-shar commented 3 years ago

Hi,

Sorry for the late reply. We have now added a testing script (src/test.py) which should allow you to make predictions.

Do I understand correctly that argmax would suit here? Yes, we have used argmax for predictions

Lastly, how should we interpret the result? Say the argmax gives value '2' does it mean that this is the highest level of empathy? As described in the paper, our empathy framework has three communication mechanisms (emotional reactions, interpretations, explorations), each having three levels (0 to 2). So, a score of 2 for emotional reactions represents its strong communication in the response.

Thanks!