QConv2DBatchnorm: propagate batchnorm parameters and fix name

google / qkeras

QKeras: a quantization deep learning library for Tensorflow Keras

Apache License 2.0

537 stars 104 forks source link

QConv2DBatchnorm: propagate batchnorm parameters and fix name #70

Closed jmduarte closed 3 years ago

jmduarte commented 3 years ago

This PR does two things

correctly propagates batchnorm parameters (I was seeing a diverging nan loss after 1 epoch without this change)
leave name unspecified so it can be automatically assigned in case the user forgets to specify (name=None actually causes an error)

zhuangh commented 3 years ago

Thanks @jmduarte

qdepthwiseconv2d_batchnorm should have the same problem, right? could you also do that in this PR, thanks.

Do you have an example that has the divergence problem? consider to add it into unittest?

jmduarte commented 3 years ago

@zhuangh thanks for checking this out!

I similarly fixed qdepthwiseconv2d_batchnorm in this PR. Unfortunately, I couldn't reproduce the original "diverging" loss behavior with the master branch, so it may just have been a bad configuration of my model originally.

In any case, I think propagating the batchnorm parameters as done in this PR still makes sense.

lishanok commented 3 years ago

LGTM