hkyoon95
commented
9 months ago The major contribution of the paper: MQS Loss, why optimizing such an objective can potentially encourage diverse generation?
Why maximizing question similarity could lead to more diverse results, wouldn't it cause less diverse cause they tend to be more similar in the representation space?
Many thanks!
Thank you for your interest in our work. You make a good point about the potential impact on diversity when making representations similar. However, there are some nuances associated with MQS Loss. This loss function aligns the representation of reference questions from the encoder with the target question's representation at the sentence level. These questions are answerable by the same context given to the encoder.
What's crucial here is the inherent difference in the representations of the encoder and decoder. This difference indicates that the encoder and decoder are processing and understanding the input in distinct ways. MQS Loss, when combined with CE (Cross-Entropy) loss, works to bridge this gap. By doing so, it encourages the model to recognize and generate a diverse range of questions that are contextually related but differently addressed.
The diversity in generation arises not simply from making the representations similar, but from the model learning to translate the diverse encoder representations into coherent decoder outputs. This process inherently involves exploring various formulations and structures of questions within the same context. Therefore, rather than reducing diversity, MQS Loss helps in generating a rich array of questions that, while semantically aligned with the given context, exhibit varied expressions and perspectives.
The major contribution of the paper: MQS Loss, why optimizing such an objective can potentially encourage diverse generation?
Why maximizing question similarity could lead to more diverse results, wouldn't it cause less diverse cause they tend to be more similar in the representation space?
Many thanks!