Different F1 scores with the same hyperparameters and same random seeds

[abdkiwan]

Hello,

I trained a knowbert language model on some medical data, and want to fine-tune it on a relation classification task. However, I get different Micro/Macro F1 scores every time a fine-tune it while keeping the same hyperparameters and the same random seeds.

Does anybody know what the problem could be ?

thanks in advance,

[MichalPitr]

Probably check if you are setting random seeds for all random number generators - Python's random, numpy, torch, sklearn, etc...