search

armbues / SiLLM

SiLLM simplifies the process of training and running Large Language Models (LLMs) on Apple Silicon by leveraging the MLX framework.
MIT License
221 stars 21 forks source link

[Feature request] Add ORPO finetuning #1

Open s-kostyaev opened 6 months ago

s-kostyaev commented 6 months ago

Hi @armbues, thank you for this great project.

Please add ORPO finetuning to do SFT and DPO in one step https://arxiv.org/abs/2403.07691

armbues commented 6 months ago

Great idea! I will add this to the roadmap.

armbues commented 6 months ago

The official implementation of the "ORPO Trainer" can be found here.