search

xgmiao / Pyramid-Attention-Networks

Pyramid Attention Networks
23 stars 10 forks source link

Wrong implementation of GAU #1

Open Venka97 opened 5 years ago

Venka97 commented 5 years ago

Hi, Quoting the GAU explanation from the paper

Our Global Attention Upsample module performs global average pooling to provide global context as a guidance of low-level features to select category localization details. In detail, we perform 3×3 convolution on the low-level features to reduce channels of feature maps from CNNs. The global context generated from high-level features is through a 1×1 convolution with batch normalization and ReLU non-linearity, then multiplied by the low-level features. Finally, high-level features are added with the weighted low-level features and upsampled gradually

Your implementation has a slight variation in implementation.

Venka97 commented 5 years ago

Previous attention modules like SEnet use sigmoid for converting the activation in the range 0-1. Does this somehow give better results?