search

microsoft / BackwardCompatibilityML

Project for open sourcing research efforts on Backward Compatibility in Machine Learning
https://backwardcompatibilityml.readthedocs.io/en/latest/
MIT License
71 stars 14 forks source link

Tensorflow New Error and Strict Imitation versions of NLLLoss, rescale dissonance #155

Open ilmarinen opened 3 years ago

ilmarinen commented 3 years ago
  1. Due to the mean reduction being used on the base loss, the dissonance is too large in relation to the base loss.
  2. The dissonance will need to be re-scaled by taking the mean over the batch.