Closed powermano closed 1 year ago
self.channel_reschedule() this module contains a LayerNorm which takes the input tensor of (B,N, C) in general vision task, image tensor shape is (BCHW), so you need to do permute channel dimension to last.
btw, layernorm has BCHW version, but it's slower.
self.channel_reschedule() this module contains a LayerNorm which takes the input tensor of (B,N, C) in general vision task, image tensor shape is (BCHW), so you need to do permute channel dimension to last.
btw, layernorm has BCHW version, but it's slower.
Thanks for your answer, I mean why is the code not directly written like this
x = x.permute(0, 3, 1, 2)
My bad, sorry.🤣
https://github.com/megvii-research/RevCol/blob/2c36531d52b985dee43f68d0c5a9735a9cba5bf5/models/modules.py#LL27C1-L27C38
I do not know why ?