search

Visual-Attention-Network / VAN-Classification

Apache License 2.0
812 stars 85 forks source link

Attention vs Add in LKA #32

Open iumyx2612 opened 1 year ago

iumyx2612 commented 1 year ago

In table 3, changing attention (mul) to add reduces VAN performance from 75.4 to 74.6. I think this is really huge. However, in the ablation study, you stated that "Besides, replacing attention with adding operation is also not achieving a lower accuracy". Is it okay to say it like that since the performance drop is 0.8

Can't treat add as a type of attention function? In Attention Mechanisms in Computer Vision: A Survey, we have the formula: image I can treat function f here is an addition operation can't I?

iumyx2612 commented 1 year ago

@MenghaoGuo Hello can you explain this?