search

z-x-yang / GCT

Gated Channel Transformation for Visual Recognition (CVPR 2020)
133 stars 26 forks source link

GCT vs ECA #2

Closed mnikitin closed 4 years ago

mnikitin commented 4 years ago

Hello!

Have you compared your GCT module with ECA (https://arxiv.org/abs/1910.03151) on some task?

Thank you!

z-x-yang commented 4 years ago

ECA is a good work, which also focuses on simplifying SE yet improving the performance. But we do this in two different ways. Notably, both of ECA and our GCT are accepted by CVPR 2020.

I haven't run the official code of ECA. But compared to the results in its paper and poster, GCT achieves comparable results on ImageNet and brings similar increments on COCO.

mnikitin commented 4 years ago

Ok, thanks! I'm going to implement GCT using mxnet/gluon, and directly compare it with ECA (already implemented) on some toy problem, like CIFAR10. If you're interested in results of such comparison, I can share it with you.

z-x-yang commented 4 years ago

I'm glad you are interested in our work. If you have more results of the comparison, I'll be happy to discuss them with you. You can also send an email to me, and I'm looking forward to it.

If you don't have more questions, I'll close this issue. Thanks!

mnikitin commented 4 years ago

Great! No more questions, closing the issue.

DietDietDiet commented 4 years ago

Ok, thanks! I'm going to implement GCT using mxnet/gluon, and directly compare it with ECA (already implemented) on some toy problem, like CIFAR10. If you're interested in results of such comparison, I can share it with you.

Hi, could you share your comparison result on GCT and ECA, or do u have a repository for it? Thanks

mnikitin commented 4 years ago

@DietDietDiet hello! I have implemented both methods using gluon: https://github.com/mnikitin/channel-attention In my experiments on cifar10 ECA demonstrates better classification accuracy than GCT, however I believe results on cifar10 are not representative. Moreover, I have compared these two methods on some facial analysis problems and found that they provide similar results on average (but different increments on different datasets). Summarizing, I would recommend to compare ECA and GCT on your task by yourself

z-x-yang commented 4 years ago

@mnikitin I've read your code and the implementation of GCT looks good.

However, it looks like that you apply GCT in the same way of SE, but this is not correct. First, we propose to apply GCT before convolutional layers, instead of after normalization layers. Second, due to GCT is much more light-weight than SE, we propose to apply GCT before every convolutional layer. However, if you want to compare GCT with ECA using a same number of GCT/ECA modules, you can apply GCT before the first 1x1 Conv in each Bottleneck ResBlock. In most of the ResBlock (except the first ResBlock of each stage), the channel number before the first 1x1 Conv is the same as the number after last 1x1 Conv.

Our TensorFlow implementation will automatically apply GCT before convolutional layers. In our PyTorch implementation, we manually apply GCT before every convolutional layer.

mnikitin commented 4 years ago

@z-x-yang thank you! That's my mistake. I will apply GCT module in the correct way and update cifar10 results

z-x-yang commented 4 years ago

@mnikitin It’s my pleasure. I'm happy to see a re-implementation in mxnet.