TensorflowTrainer improvements - Githubissues

capreolus-ir / capreolus

A toolkit for end-to-end neural ad hoc retrieval

https://capreolus.ai

Apache License 2.0

95 stars 32 forks source link

TensorflowTrainer improvements #121

Closed andrewyates closed 3 years ago

andrewyates commented 3 years ago

from @crystina-z's branch:

change the meaning of itersize in tensorflow trainer from "number of steps" to "number of instance" (as mentioned in #117 )
add fast-forward training to tensorflow trainer (review required)
rename warmupsteps -> warmupiters, decaystep -> decayiters
change the criteria of determinating "bert" variable, to also include "electra"

Additional changes:

Updated itersize in docs/reproduction to keep behavior the same
Reverted to previous linear decay formula to keep behavior the same. (Also: new one had a bug that could make LR negative.)
Added reranker.trainer.amp config option defaulting to false (same as pytorch)
Fix TF loss averaging in logging statement, which was also broken previously
Add previous tfrecord fixes (@crystina-z can you check these to make sure I merged correctly?)