search

Unbabel / OpenKiwi

Open-Source Machine Translation Quality Estimation in PyTorch
https://unbabel.github.io/OpenKiwi/
GNU Affero General Public License v3.0
229 stars 48 forks source link

Add saving of latest model during training and improved error handling of resuming #36

Closed captainvera closed 5 years ago

captainvera commented 5 years ago

OpenKiwi now saves the latest model at every epoch. This, while maintaining previous checkpointing behaviour.

At checkpoint time Kiwi checks if this is the best eval so far. If not it saves the full model under output-dir/latest_epoch. Else, it saves the best eval as previously and places links to those in the aforementioned folder.

On resuming, Kiwi now looks for output-dir/latest_epoch. If this is not found it falls back into previous behaviour (loading from checkpoints) as such, this should be 100% compatible with previous kiwi versions.

This came as a response to unclear behaviour, brought up in #35.

captainvera commented 5 years ago

All comments have been taken into account. Waiting for review