We found that SyncBatchNormalizationCuda gave an incorrect outputs in channel last layout. It's a test to check if the problem is fixed in sony/nnabla-ext-cuda#457.
EDIT:
It turned out that gradient values of gamma do not match between memory layouts. We defer the investigation in order to merge the change for the first issue we've found and resolved.
We found that SyncBatchNormalizationCuda gave an incorrect outputs in channel last layout. It's a test to check if the problem is fixed in sony/nnabla-ext-cuda#457.
EDIT:
It turned out that gradient values of gamma do not match between memory layouts. We defer the investigation in order to merge the change for the first issue we've found and resolved.