search

capreolus-ir / capreolus

A toolkit for end-to-end neural ad hoc retrieval
https://capreolus.ai
Apache License 2.0
95 stars 32 forks source link

Reproduction of model training #201

Closed lingwei-gu closed 2 years ago

lingwei-gu commented 2 years ago

Submitted the job through SLURM on Compute Canada.

One major issue for me in the sample_slurm_script.sh was: After loading module StdEnv/2018.3, module java/11 became inactive and cannot be found anymore. Thus, I looked up the documentations and changed it to StdEnv/2020, then it works.

crystina-z commented 2 years ago

@lingwei-gu hi lingwei! Thanks for helping updating the repro doc. tho the monobert r expected to hv MRR@10 to be 0.35+. chould u chime in the issue #200 and see if the decay line is also missing from the script u were using as well?

lingwei-gu commented 2 years ago

@crystina-z You are right it is missing reranker.trainer.decay=0.1 in this file: config_msmarco.txt I'll run the experiment again and see how it goes

d1shs0ap commented 2 years ago

@crystina-z The decay line was no longer missing after using the latest nimasadri11/capreolus repository.