Issue about the channel attention.

NVlabs / FAN

Official PyTorch implementation of Fully Attentional Networks

https://arxiv.org/abs/2204.12451

Other

469 stars 28 forks source link

Issue about the channel attention. #22

Open night3759 opened 1 year ago

night3759 commented 1 year ago

Thank you for you great job. When I read the paper, I am confused about the "Attentional feature transformation" part. I don't know why the channel dimension attention can stand for feature covariance. And I didn't find the relating code about equation (6). I look forward to receiving your reply. Thank you.

zhoudaquan commented 1 year ago

Hi, thanks for your interest in the work, and sorry for the late reply.... Eqn(6) is not used as it was replaced by Eqn(7) in the final implementation. However, it is easy to reproduce simply by manipulating the original SA module by swapping the dimension of the formula of self-attention calculation from NC x CN --> CN x NC and then matmal with CN dimensioned v vector.

I hope this clarifies your questions. If you have any other questions, do drop me a message.