search

ZK-Zhou / spikformer

ICLR 2023, Spikformer: When Spiking Neural Network Meets Transformer
MIT License
277 stars 43 forks source link

Confused about the use of layernormalization in your Imagenet training code. #14

Open WANGXinuuu1 opened 10 months ago

WANGXinuuu1 commented 10 months ago

Thanks for your kind sharing for your codes. You did a really good and meaningful research work in the field of SNN.

I just confused about # norm_layer=nn.LayerNorm # in your code of training Imagenet.

But in your paper, I found the state "We empircally find that the layer normalization (LN) does not apply to SNNs, so we use batch normalization (BN) instead." It seems a little bit inconsistent.

Many thanks!

Xin Wang