The sentiments are varying every time when i run it for the same sentence.

yya518 / FinBERT

A Pretrained BERT Model for Financial Communications. https://arxiv.org/abs/2006.08097

Apache License 2.0

560 stars 128 forks source link

The sentiments are varying every time when i run it for the same sentence. #15

Closed man0007 closed 3 years ago

man0007 commented 3 years ago

I have created my own sentence "there is a abundance of capital because of good profits" to test the sentiment prediction for the fine-tuned model using the notebook given: "FinBert Model Example.ipynb".

When I ran for the first time I got the result as 'positive': https://prnt.sc/vf3fnb When I ran for the second time I got the result as 'neutral': https://prnt.sc/vf3fcr

Why is this variation happening? How can we trust the results from this model if it is not stable?

yya518 commented 3 years ago

Thanks for reporting this. There should be model.eval() in the notebook. It's fixed now.

m1996 commented 3 years ago

Hi I am facing the same problem. I just commented model.load_state_dict(torch.load(fine_tuned_weight_path)) ...