Chacha-Chen / MPLight

56 stars 18 forks source link

confusion about the multi-head attention #11

Open ynuwm opened 3 years ago

ynuwm commented 3 years ago

In the function GAT_net in stgat_agent.py, why you only return the last relation in line 544:

return relation_pool[-1], att_record_all_layers

If we set two layers of MultiHeadsAttModel, according this code it only return the last relation, I think we should weight them and add them up. Can you give me an explanation? Thank you.