Which random seed are you using?

DreamInvoker / GAIN

Source code for EMNLP 2020 paper: Double Graph Based Reasoning for Document-level Relation Extraction

MIT License

142 stars 31 forks source link

Which random seed are you using? #19

Open yiqingxyq opened 3 years ago

yiqingxyq commented 3 years ago

Hi, I'm trying to reproduce the results on BERT-base, but I can only get F1 / ignF1 = 0.5985 / 0.5752 for full model, and F1 / ignF1 = 0.6010 / 0.5796 for the nomention ablation.

I am using the given .sh so I suppose the reason is not the hyper-parameters. Could you provide the random seed you are using to reproduce the F1 = 0.6122 result? Thx!

HenryWang628 commented 3 years ago

I met the same situation, I tried to reproduce the results on BERT-base, but I got F1 = 0.6017 and AUC = 0.5763 for full model after training for over 100 epoches. I also find that the train_loss start rising after about 90 epoches, the hyper-parameters are totally the same as that in the paper. I don't know whether there was anything wrong with my process.

Ber666 commented 3 years ago

My result of bert-base is similar to yours.

logan-markewich commented 3 years ago

I am also running into the same problem with bert-base. bert-large was too big to train on my GPU.

It would be helpful if the authors could post the pretrained models, I created another issue for this