thunlp / OpenPrompt

An Open-Source Framework for Prompt-Learning.
https://thunlp.github.io/OpenPrompt/
Apache License 2.0
4.38k stars 455 forks source link

Question about the test score in tutorial/3.1_lmbff.py #227

Closed ZekaiShao25 closed 1 year ago

ZekaiShao25 commented 1 year ago

Hello, I ran this tutorial code implementing lmbff and the final test score was 0.5091743119266054.

This performance is too low, so I checked the evaluation score during training and the prediction results. In fact, the model scored 0.5 in each epoch, and the final prediction was all 0. Exactly, 0.509 equals to the score predicting majority as reported in the paper of lmbff.

I did not check in more detail, but apparently there is something wrong with the training of the model here.

nirajpahari commented 1 year ago

I also ran into the exactly same problem. All the predictions are for the same class.