search

deephealthproject / eddl

European Distributed Deep Learning (EDDL) library. A general-purpose library initially developed to cover deep learning needs in healthcare use cases within the DeepHealth project.
https://deephealthproject.github.io/eddl/
MIT License
34 stars 10 forks source link

ONNX loader doesn't load the BatchNorm parameters `epsilon` and `momentum`? #305

Closed lauracanalini closed 2 years ago

lauracanalini commented 2 years ago

To Reproduce Steps to reproduce the behavior:

  1. Load ResNet ONNX here https://drive.google.com/file/d/1gqLI0k4NEQL6dsZFx-iI2zFZBBgucUTd/view?usp=sharing
  2. Check values in layers[2]
  3. Open ONNX in netron.app
  4. Compare values

Expected behavior They should be the same.

chavicoski commented 2 years ago

They are almost the same, there is a certain loss of precision.

Reproducing the issue: Netron: momentum = 0.009999999776482582 - EDDL momentum = 0.01 Netron: epsilon = 0.10000000149011612 - EDDL epsilon = 0.1

And they must be loaded because the default values in ONNX are momentum=0.9 and epsilon=1e-5.

https://github.com/deephealthproject/eddl/blob/e6de5aaf5cf6308174ce40c3147ecd1b77a05357/src/serialization/onnx/net/layers/normalization/batchnorm_onnx.cpp#L13

lauracanalini commented 2 years ago

Right, I was loading the ONNX incorrectly. But now I have another question: if as reported issue #302, eddl uses 1 - momentum compared to PyTorch, when it loads a PyTorch ONNX shouldn't it load 1 - onnx momentum?

chavicoski commented 2 years ago

Yes, you are right. I am going to change it.

chavicoski commented 2 years ago

I checked the ONNX operators definition and it is correct now, I don't have to change it.

You can see here that the ONNX definiton uses the momentum like the EDDL. This means that if you use a momentum of 0.1 in Pytorch, It should export it as momenum=0.9. But the EDDL is importing it right.