about generation loss

[bjtuxck]

src/modeling code: if self.training:

只对positive计算generation loss

 # generated_q_hidden = generated_q_hidden[:,None,:,:].view(bz, self.args.sample_num, -1, p_hidden.size(-1))[:,0,:,:]  # [bz, seq_len, 768]
 if mlm_labels is not None and mlm_labels['decoder_mlm_labels'] is not None:
       mlm_loss += self.mlm_loss(generated_q_hidden, mlm_labels['decoder_mlm_labels'])  # query生成loss

如何保证只对【正样本】计算mlm_loss？负样本是如何过滤的？

[Deriq-Qian-Dong]

p_hidden的第一个位置都是正样本的hidden state，只用第一个就行了 参考代码中的p_hidden.size(-1))[:,0,:,:]