Closed orangetwo closed 3 years ago
class MultiHeadAttention(nn.Module):
in this class, it does not implement the scale of the multiplication of Query and Key. and in the forward function, it seems that the funcation should return linear_proj , not output?
@orangetwo Thanks for spotting this. I've checked in the fix. Feel free to close this once you've confirmed it.
class MultiHeadAttention(nn.Module):
in this class, it does not implement the scale of the multiplication of Query and Key. and in the forward function, it seems that the funcation should return linear_proj , not output?