Incorrect prediction

[Abhijith002]

Hi,

Used pre-trained models on financial news of a company for text classification. Given below are the three sentences which should have been predicted as positive. Is there a more refined model available or we need to further fine tune it ourselves ?

Thanks,

[prats0599]

I'm facing the same issue. However the same model on this link seems to perform well. Did you find a solution?

[Abhijith002]

I did try to run the prediction using the model that is there on huggingface, but it did not give me any better result. I did not find any solution so I am currently gathering the data for training.

[prats0599]

I tested the model on huggingface and it seems to be working fine now. All my predictions for the test set were negative which led me to believe something was wrong w my code. I finally got it to work with some tweaks to the predict function in this repo here. I also used the Finbert Tokenizer("ProsusAI/finbert") instead of ''bert-base-uncased''.

[Abhijith002]

The prediction is going wrong particularly with statements l gave in the screenshot above. The statements like "Improving free cash flows has meant that the company's net debt also continues to decline" is a positive scenario for a company since there is decline in debt. But because of the word "decline" it is being predicted as negative. There are 20 statements above and below those three which were predicted correctly.

[doguaraci]

Hi, we don't have any other model. The model might make incorrect predictions. As reported on the paper, the accuracy on the test set was 86%.

With regards to @prats0599's problem of getting always negative predictions, that obviously shouldn't happen. @prats0599 could you please share more details about the problem and the fix you did?