Open astariul opened 4 years ago
I'm confused about one detail in your paper.
From Figure 1, it seems the contrastive attention mecanism is applied to each decoder layer :
But from the text, it is mentioned that each attention head have different responsability and you hand-picked the best one :
So I'm not sure if the contrastive mecanism is applied to every head of the model or only the one you chose.
I'm confused about one detail in your paper.
From Figure 1, it seems the contrastive attention mecanism is applied to each decoder layer :
But from the text, it is mentioned that each attention head have different responsability and you hand-picked the best one :
So I'm not sure if the contrastive mecanism is applied to every head of the model or only the one you chose.