search

ygean / SE_DenseNet

This is a SE_DenseNet which contains a senet (Squeeze-and-Excitation Networks by Jie Hu, Li Shen, and Gang Sun) module, written in Pytorch, train, and eval codes have been released.
https://zhuanlan.zhihu.com/p/48499356
MIT License
100 stars 18 forks source link

More variant architecture of se_densnet need to try and test #1

Open John1231983 opened 5 years ago

John1231983 commented 5 years ago

Hi, thanks for sharing your experiment results. I checked and found that you may have some redundant code in the _Dense layer that adds thr seblock in of convolution. You added it in loop (for) and after first convolution. Why do you add seblock in _Dense_layer again? Thanks

ygean commented 5 years ago

@John1231983 Hi, I found redundant code which you pointed, I think it's my mistake. For testing se_densenet, I have write some scripts to test se_densenet on cifar10 datasset today. And I will test whether redundant code removed can influence train-test result.Let's wait a few days to see result, and I will update README in a week.

John1231983 commented 5 years ago

Good. And consider to remove seblock after transition block also. We often use seblock after denseblock only. Transition block only helps to reduce feature size

ygean commented 5 years ago

@John1231983 Yes, thank you for your suggestions, I will take it and make more comparative experiments.

ygean commented 5 years ago

@John1231983 Hi, John. I update my test result just now, pls check it. Thank you very much.

John1231983 commented 5 years ago

Good job, But the result shows that with and without seblock has similar performance. In full code, you have added the sebock in Trans layer and Dense layer. How about add them in the loop function and remove in the _Transition and _Dense layer? I mean add in https://github.com/zhouyuangan/SE_DenseNet/blob/e5bb2fda0e49d50c274e29330c0948d3aa4a4376/se_densenet_full.py#L209

and https://github.com/zhouyuangan/SE_DenseNet/blob/e5bb2fda0e49d50c274e29330c0948d3aa4a4376/se_densenet_full.py#L219

And remove it in https://github.com/zhouyuangan/SE_DenseNet/blob/e5bb2fda0e49d50c274e29330c0948d3aa4a4376/se_densenet_full.py#L167

https://github.com/zhouyuangan/SE_DenseNet/blob/e5bb2fda0e49d50c274e29330c0948d3aa4a4376/se_densenet_full.py#L137

Thanks

ygean commented 5 years ago

@John1231983 It's worthy to make a test, I will update new result after job done, pls keep watching.Thanks.

ygean commented 5 years ago

The new test result has updated. I will release train and test code in a few days.

John1231983 commented 5 years ago

Good. That is what I expect. You also can try something as:

  1. Sebock in loop only after denseblock ( remove seblock in transittion)
  2. Seblock in loop only after transition block ( remove seblock in denseblock)

Actually, we do not know where seblock will be good for densenet architecture. In my opinion, the first case may get better result