About SE layer in Fused-MBConv

d-li14 / efficientnetv2.pytorch

PyTorch implementation of EfficientNetV2 family

https://arxiv.org/abs/2104.00298

MIT License

460 stars 93 forks source link

About SE layer in Fused-MBConv #5

Closed Noir97 closed 3 years ago

Noir97 commented 3 years ago

I understand that the SE layer might get removed in the original fusion attempt: https://ai.googleblog.com/2019/08/efficientnet-edgetpu-creating.html

But according to Fig2 in the paper, it seems like the SE layer is preserved in the Fused MBConv. So I am quite confused when I found the SE layer lacking in your implementation. Have you ever tried the implementation with SE layer, or am I missing something in the paper?

Noir97 commented 3 years ago

OK...I guess the missing SE parameter in the architecture table implies that. Nevermind.

xiaoerlaigeid commented 3 years ago

OK...I guess the missing SE parameter in the architecture table implies that. Nevermind.

I also notice this implementation lack of SE layer in Fused MBConv. Could further explain about this ?

Noir97 commented 3 years ago

OK...I guess the missing SE parameter in the architecture table implies that. Nevermind.

I also notice this implementation lack of SE layer in Fused MBConv. Could further explain about this ?

In the architecture table, "SE" shows up but only with normal MBConv, I guess that means it is not used with Fused MBConv

xiaoerlaigeid commented 3 years ago

Oh thanks！