Enables multi-GPU Batch Norm, connects training of Batch Norm parameters to current optimizer, and APIifies incoming (pre-BN) versus outgoing (post-BN) units and deltas.

amazon-archives / amazon-dsstne

Deep Scalable Sparse Tensor Network Engine (DSSTNE) is an Amazon developed library for building Deep Learning (DL) machine learning (ML) models

Apache License 2.0

4.41k stars 731 forks source link

Closed scottlegrand closed 5 years ago

scottlegrand commented 5 years ago

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

slegrandA9 commented 5 years ago

Do not approve this pull request yet. Edward and I need to make sure it's correct. It may indeed already be so, but we need to make sure...

ekandrotA9 commented 5 years ago

Reviewed the code and reran cifar-10 with Batch Norm - all looked good.