ReFT + DPO Tutorial - Githubissues

stanfordnlp / pyreft

ReFT: Representation Finetuning for Language Models

https://arxiv.org/abs/2404.03592

Apache License 2.0

947 stars 77 forks source link

Closed AmirZur closed 2 months ago

AmirZur commented 2 months ago

Introduced tutorial for ReFT + DPO in examples/dpo folder.

Files:

examples/dpo/dpo_trainer.py - adapting DPOTrainer from trl library to ReFT models.
examples/dpo/dpo.ipynb - tutorial notebook for ReFT + DPO on TruthfulQA dataset.
examples/dpo/README.md - overview file for tutorial.

frankaging commented 2 months ago

Thanks!