microsoft / Relation-Aware-Global-Attention-Networks

We design an effective Relation-Aware Global Attention (RGA) module for CNNs to globally infer the attention.
MIT License
341 stars 65 forks source link

What's the difference between RGA and ordinary attention module? #16

Open TheBobbyliu opened 3 years ago

TheBobbyliu commented 3 years ago

It seems like RGA module is different from ordinary attention module, like the one used in ABD-Net. In your article, the main advantage of RGA is that we can extract global relation information with less computational costs. However, spatial attention module and channel attention module in ABD-Net seem to be more efficient since there's no extra computational costs on feature embedding. Could you tell me what's the advantage of RGA over ordinary attention module?

att

20200811201340578