Westlake-AI / MogaNet

[ICLR 2024] MogaNet: Efficient Multi-order Gated Aggregation Network
https://arxiv.org/abs/2211.03295
Apache License 2.0
162 stars 13 forks source link

Questions about the application scenarios of the CA module #21

Closed Zhangyuhaoo closed 1 month ago

Zhangyuhaoo commented 1 month ago

Sorry to bother you. Hello, I have read your paper and found it very impressive. I have a small question: can I use the CA module you proposed to replace the FFN layer in ViT? Again, I apologize for my interruption and look forward to your reply and suggestions. Thank you!

I hope this helps! If you need any further assistance, feel free to ask.