search

facebookresearch / MUSE

A library for Multilingual Unsupervised or Supervised word Embeddings
Other
3.18k stars 544 forks source link

Problem with saving best model with validation metric in refinement udpate #141

Open sbmaruf opened 4 years ago

sbmaruf commented 4 years ago

In refinement iterations, the best model is still selected by the validation metric, but I think in Procrustes update the result will never be decreased. So we should always take the latest updated model after Procrustes update. Related reference: https://github.com/facebookresearch/MUSE/blob/master/unsupervised.py#L179 https://github.com/facebookresearch/MUSE/blob/master/src/trainer.py#L215

@glample @aconneau [log] https://paste.ubuntu.com/p/SJ23FhpWMT/ See in the log. The best model saved whose csls_knn_10 - Precision at k = 1 is 70.133 whereas the best translation model have a score of 78.733