number9473 / nn-algorithm

algorithm for neural network
259 stars 59 forks source link

Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer #31

Open joyhuang9473 opened 7 years ago

joyhuang9473 commented 7 years ago

Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer

We show that activation-based attention transfer gives better improvements than full activation transfer, and can be combined with knowledge distillation

joyhuang9473 commented 7 years ago

10 #12