Description of changes:
Added Batch Norm code to the conv layers. So, it now works in both fully
connected and Conv.
In doing that, cleaned up the previously checked in BN code to handle
both types of BN.
Tested that the non-BN code still works, the previously Fully Connected
still works, and the the new Conv code works.
One issue - when setting a Conv layer to BN, after 10 epochs, it looks like
it starts to diverge. This seems to be due to a change in Learning Rate - one
needs to adjust ones learning rate when using BN.
So I'm adding this code, the calls are correct and I didn't find any bugs. One
might have to make adjustments to properly use BN (adjusting LR, etc).
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Issue #, if available:
Description of changes: Added Batch Norm code to the conv layers. So, it now works in both fully connected and Conv. In doing that, cleaned up the previously checked in BN code to handle both types of BN.
Tested that the non-BN code still works, the previously Fully Connected still works, and the the new Conv code works.
One issue - when setting a Conv layer to BN, after 10 epochs, it looks like it starts to diverge. This seems to be due to a change in Learning Rate - one needs to adjust ones learning rate when using BN.
So I'm adding this code, the calls are correct and I didn't find any bugs. One might have to make adjustments to properly use BN (adjusting LR, etc).
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.