megvii-research / RevCol

Official Code of Paper "Reversible Column Networks" "RevColv2"
Apache License 2.0
249 stars 10 forks source link

code issue #12

Closed powermano closed 1 year ago

powermano commented 1 year ago

https://github.com/megvii-research/RevCol/blob/2c36531d52b985dee43f68d0c5a9735a9cba5bf5/models/modules.py#LL27C1-L27C38

    x = x = x.permute(0, 3, 1, 2)

I do not know why ?

nightsnack commented 1 year ago

self.channel_reschedule() this module contains a LayerNorm which takes the input tensor of (B,N, C) in general vision task, image tensor shape is (BCHW), so you need to do permute channel dimension to last.

btw, layernorm has BCHW version, but it's slower.

powermano commented 1 year ago

self.channel_reschedule() this module contains a LayerNorm which takes the input tensor of (B,N, C) in general vision task, image tensor shape is (BCHW), so you need to do permute channel dimension to last.

btw, layernorm has BCHW version, but it's slower.

Thanks for your answer, I mean why is the code not directly written like this

x = x.permute(0, 3, 1, 2)
nightsnack commented 1 year ago

My bad, sorry.🤣