hengyicai / ContrastiveLearning4Dialogue

The codebase for "Group-wise Contrastive Learning for Neural Dialogue Generation" (Cai et al., Findings of EMNLP 2020)
MIT License
53 stars 5 forks source link

增益貌似是match-model带来的?不太明白为何对 生成多样性 有提升?多谢多谢! #1

Closed guotong1988 closed 4 years ago

guotong1988 commented 4 years ago

@HayesTsai @hengyicai 多谢多谢!

hengyicai commented 4 years ago

Hi, thanks for your attention, and we would like to clarify that:

  1. The proposed group-wise contrastive learning framework explicitly explores multiple variants of a given dialogue example and encourages distinctiveness via Eq.(1), in which the target dialogue model is expected to give higher conditional probabilities for the positives and vice versa for any negative pair, compared to the reference model.
  2. The matching model is introduced to organize a group of high-quality positive/negative examples for the contrastive loss in Eq.(4), which are critical for the contrastive learning paradigm, and we can also get rid of such matching model by using other sampling strategies like in (Gao et al., 2020) to build the positive/negative samples.

Ref: Xiang Gao, et al., Dialogue Response Ranking Training with Large-Scale Human Feedback Data. In EMNLP 2020.

guotong1988 commented 4 years ago

似乎理解了,group之间的distinctiveness增加了,?

guotong1988 commented 4 years ago

image 哦,Eq.(1)就是encourages distinctiveness