Kipok / NeMo-Skills

A pipeline to improve skills of large language models
https://kipok.github.io/NeMo-Skills/
Apache License 2.0
185 stars 41 forks source link

Add an option to prepare data in the DPO format (or a separate script) #160

Open Kipok opened 1 month ago

Kipok commented 1 month ago

Currently we only automate SFT data preparation, but should include DPO as well, since we support it for training