search

monologg / GoEmotions-pytorch

Pytorch Implementation of GoEmotions 😍😢😱
Apache License 2.0
151 stars 46 forks source link

Sigmoid or Softmax missing in evaluation #5

Closed shakib-root closed 3 years ago

shakib-root commented 3 years ago

Shouldn't sigmoid(multilabel classification) or softmax(multiclass classification) be applied here to get the predictions? Otherwise, later the thresholds are being applied on the logits directly.

https://github.com/monologg/GoEmotions-pytorch/blob/47f5afbbd1b49401e3204c8b1f228e21bfbd6ee0/run_goemotions.py#L183

monologg commented 3 years ago

@shakib-root

It's totally true that there is a mistake.

I'm fixing the issue on this branch (https://github.com/monologg/GoEmotions-pytorch/commit/d47dfed557bd847561b12d9506fa4a7bf17ad928) and re-running the models. As training is over, I'll reupload the model & result!

Thank you for informing the issue

blackcat84 commented 3 years ago

Does this affect the already fine-tuned models (like the ones on the model hub of Hugging Face) or is it just a matter of metrics computation while fine-tuning?

monologg commented 3 years ago

@blackcat84

This one doesn't affect the fine-tuned models, but there might be a possibility that the model from huggingface model hub isn't the best model cuz I've chose the model based on the wrong metrics

blackcat84 commented 3 years ago

Right, makes perfectly sense. Looking forward to your new results! Thanks!

monologg commented 3 years ago

Fixed on b75f9d3a8b76bc1060687925b04ca98a1b8ea66b :)

Also huggingface s3 models are updated!