search

UKPLab / gpl

Powerful unsupervised domain adaptation method for dense retrieval. Requires only unlabeled corpus and yields massive improvement: "GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval" https://arxiv.org/abs/2112.07577
Apache License 2.0
315 stars 39 forks source link

Evaluation every 10k steps #33

Open adrienohana opened 1 year ago

adrienohana commented 1 year ago

Hello,

I can't figure out how to use the evaluation while training, not sure what the data format is and how to plot the ndcg@k. I've figured out how to do it after training though, by loading the saved models in a loop and predicting on my evaluation data.

The question I have is, does seeing the evaluation only every 10k steps make sense ? How to be sure there aren't some big variations in between ? My training doesn't stop improving after 100k steps and is not as smooth as in your paper. Screenshot 2023-01-31 at 14 31 01

Any hints would be greatly appreciated.

shreyasavant commented 11 months ago

@nreimers @kwang2049 Would love your help with this. Need this for my implementation as well!

KuijpersNick0 commented 2 months ago

@adrienohana Did you find any way to evaluate during training ?