search

Res2Net / Res2Net-PretrainedModels

(ImageNet pretrained models) The official pytorch implemention of the TPAMI paper "Res2Net: A New Multi-scale Backbone Architecture"
https://mmcheng.net/res2net/
1.07k stars 215 forks source link

Squeeze excite module? #16

Closed chris-ha458 closed 4 years ago

chris-ha458 commented 4 years ago

The original paper makes use of squeeze and excite modules(se modules). Certain alternative implementations(https://github.com/4uiiurz1/pytorch-res2net) include them too.

However this repo does not. Are there any plans to release official code with the se modules included?

gasvn commented 4 years ago

Since se module is not a standard module in common backbone networks such as ResNet and ResNeXt. To make our backbone network simpler to use, so we didn't include se module. The se module can be inserted before the short connection of the Res2Net module.

PS: We will release a improved version of Res2Net within the next few weeks. It has more than 2% improvement on ImageNet top-1 acc. (Without using se block and other extra blocks.)

gasvn commented 4 years ago

If you have trouble in implementing the se block, I can provide you with the example code.

chris-ha458 commented 4 years ago

Thank you very much for the offer! But it is unnecessary. I primarily use the pytorchimagemodels implementation (https://github.com/rwightman/pytorch-image-models) At first thought that that code base did not implement se so I came here for an official implementation. Recently I found out that pytorch image models has indeed implemented se module that can be easily used with res2net.

After seeing assembled cnn which used resnet base model to exceed efficientnets in many metrics, I am trying to add res2net into the assembled cnns.

Anyway I am eager to see your new improved model! Since it achieves better performance without se, it would no doubt be possible to achieve even better results with se or other channel attention modules. I thank you for your effort and look forward to your release!

gasvn commented 4 years ago

Res2Net_v1b is now released. Hope you can get better performance with our newly released models.

chris-ha458 commented 4 years ago

The new results are very impressive! However, the paper does not seem to reflect this new changed model. Is it the same network but with newer datasets augmentation or training? More information and reproduction information would definitely helpful!

gasvn commented 4 years ago

We have added some tricks to improve the network structure. And we use the training method we are working on now to get a better performance. We will release the code once the work is done.