search

tengshaofeng / ResidualAttentionNetwork-pytorch

a pytorch code about Residual Attention Network. This code is based on two projects from
667 stars 165 forks source link

Expression of mix attention #22

Open fengshenfeilian opened 5 years ago

fengshenfeilian commented 5 years ago

Thanks for your job! I have a question about the expression of mix attention. And is conv->relu->conv->sigmoid able to represent it?

yanjj16 commented 3 years ago

I have just the same question.The orginal paper said ,for Mixup attention,they use the sigmoid function as the activate function, Apparently,this implementation use relu function instead.

yanjj16 commented 3 years ago

In fact, as a freshman,I'm a little confused about the mixup_criterion function in the train_mixup.py file,could you give me some instruction about it?