search

sail-sg / poolformer

PoolFormer: MetaFormer Is Actually What You Need for Vision (CVPR 2022 Oral)
https://arxiv.org/abs/2111.11418
Apache License 2.0
1.3k stars 117 forks source link

Inquiry about the Hybrid design #32

Closed AbdelrahmanShakerYousef closed 2 years ago

AbdelrahmanShakerYousef commented 2 years ago

Hello, thank you for sharing the code of the paper.

Could you please release the code of the hybrid design?

Also, I have a question please about the hybrid design in Table. 6. Did you replace the whole pooling stage with attention or SpatialFC stage ? or just replace the last block of each stage?

Thank you.

yuweihao commented 2 years ago

Hi @AbdelrahmanShakerYousef ,

Thanks for your attention.

Sorry that currently, I need to prepare for my final exams at university. I will release the code and trained models of the hybrid design before 1 May. We replace the whole pooling stage with the attention or SpatialFC stage. Remember to add position embedding before the first attention stage. For hybrid design, the norm is set as Layer Norm and the batch size as 1024.

yuweihao commented 2 years ago

Hi @AbdelrahmanShakerYousef ,

I have completed the release of code and trained models of the hybrid design before 1 May (Anywhere on Earth Time Zone haha).

Thanks for your attention :)