search

xfactlab / orpo

Official repository for ORPO
Apache License 2.0
412 stars 38 forks source link

how to do ORPO with ShareGPT data? #11

Closed pabl-o-ce closed 5 months ago

pabl-o-ce commented 5 months ago

is there a way to modify a current dataset ShareGPT to ORPO?

nlee-208 commented 5 months ago

Hi @pabl-o-ce,

If you're talking about the ShareGPT_Vicuna_unfiltered dataset, it does not have a pairwise preference format necessary for ORPO. So in that case you can further develop pairs based on the dataset or maybe use it to perform SFT before applying ORPO.

pabl-o-ce commented 5 months ago

Hi @nlee-208 yeah the main problem I was looking is to generate the negative response for complete a dataset that is on sharegpt