how to visualize network with BN?

[baodingge]

my network was trained from scratch, so i use some BN layer.Should I modify the BN layer gradient?

[falaktheoptimist]

Hi @baodingge Yes, you're right - you'll need to modify the gradient for batch normalization. We have included a case for layer normalization but not for batch normalization.

However, we shuould have a similar case for batch normalization too - one that simply passes back the gradient that comes in from the succeeding layer to the preceding one. If you do make a change in tf_cnnvis, send us a PR and we'll merge or, we'll add the required case in a few days.

Also, another plan would be to try out reconstruction without any change to the gradients, which would be a bit grey, but my guess is that it should work fine too. Let us know how it works out.

Thanks!

[baodingge]

Yes, I use the code without change the code, it also works well, and it looks a bit grey,but it is useful.

[falaktheoptimist]

Alright. Thanks for the update. Applying Histogram equalization on the above image might get you an even better output.