Pytorch supports PARADE, AMP, and LR scheduler

capreolus-ir / capreolus

A toolkit for end-to-end neural ad hoc retrieval

https://capreolus.ai

Apache License 2.0

95 stars 32 forks source link

Closed andrewyates closed 3 years ago

andrewyates commented 3 years ago

Ports PARADE (transformer) model to Pytorch. Other variants not yet supported.
Adds LR scheduling config options to Pytorch mirroring TF's.
Adds AMP support to Pytorch
Fixes seeding bug when enabling Pytorch multithreading
Pytorch saves weights from every epoch only when fastforwarding is enabled (best dev weights always saved)

Closes #86 and closes #87.