Update default values for batchnorm

google / objax

Apache License 2.0

768 stars 77 forks source link

Update default values for batchnorm #242

Open yechengxi opened 2 years ago

yechengxi commented 2 years ago

change default batch norm momentum and eps to match the pytorch implementation. The previous setting gave us a slow update speed.

aterzis-google commented 2 years ago

Thanks for the PR Chengxi!

It's not one of Objax's goals to mirror PyTorch so it's OK to have different defaults. Also we would like to see more evidence that the new default values are better across a wide variety of settings.