search

vlawhern / arl-eegmodels

This is the Army Research Laboratory (ARL) EEGModels Project: A Collection of Convolutional Neural Network (CNN) models for EEG signal classification, using Keras and Tensorflow
Other
1.14k stars 284 forks source link

Batch normalization layers #39

Closed Sara04 closed 2 years ago

Sara04 commented 2 years ago

Hello vlawhern,

I really appreciate that you have implemented all these models in tensorflow and put them available here. They are really easy to use and it saved me a lot of time.

I have only one remark. I've noticed that different libraries (tensorflow/torch etc) use momentum in batch normalization differently (in tensorflow momentum of 0.9 corresponds to 0.1 in torch). I've checked the code of the authors of DeepConvNet and ShallowConvNet (https://github.com/robintibor/braindecode) and they are using torch, so the default momentum in tensorflow should be 0.9.

Best regards,

Sara

vlawhern commented 2 years ago

Hi Sara

Thanks for reaching out and posting this. After doing some homework on my end I can see that TF and PyTorch define momentum differently (it being a momentum vs 1-momentum thing it seems).. I've pushed the change to the repo (https://github.com/vlawhern/arl-eegmodels/commit/4a512e503198db2010848813ead9afbf8cd54c97).

If there are any other issues please let me know.