golsun / DialogRPT

EMNLP 2020: "Dialogue Response Ranking Training with Large-Scale Human Feedback Data"
MIT License
336 stars 33 forks source link

How context part is constructed? #2

Closed Rexhaif closed 3 years ago

Rexhaif commented 3 years ago

Hi, I've read the paper and repository description. But I'm still unsure how the context part of model input is constructed. So far I see two possible options:

golsun commented 3 years ago

Hi @Rexhaif

DialogRPT supports multi-turn context, if you're using the latest version I just updated (this commit), the context should look like:

 'context_turn_1' <|endoftext|> 'context_turn_2' <|endoftext|> 'context_turn_3' <|endoftext|> ...<|endoftext|> 'context_turn_4'

It's consistent with DialoGPT and Hugging Face model card (see example)

Rexhaif commented 3 years ago

Thanks!

Mikeygoldman1 commented 11 months ago

@golsun Hi, in this case below, is the model ranking 'context_turn_4' while taking into account all the previous contexts?

'context_turn_1' <|endoftext|> 'context_turn_2' <|endoftext|> 'context_turn_3' <|endoftext|> ...<|endoftext|> 'context_turn_4'