Open 2578440997 opened 8 months ago
queries = self.query_projection(queries).view(B, L, H, -1) keys = self.key_projection(keys).view(B, S, H, -1) values = self.value_projection(values).view(B, S, H, -1) out, attn = self.inner_attention( queries, keys, values, attn_mask ) if self.mix: #此时out.shape = 32,96,8,72 out = out.transpose(2, 1).contiguous() #此时out.shape = 32,8,96,72 out = out.view(B, L, -1) #此时out.shape = 32,72,512 return self.out_projection(out), attn
关于解码器的self.mix为True时, 此时out.shape = 32,96,8,72, 为什么要将变成32,8,96,72呢?
同问
关于解码器的self.mix为True时, 此时out.shape = 32,96,8,72, 为什么要将变成32,8,96,72呢?