search

daniel-furman / sft-demos

Lightweight demos for finetuning LLMs. Powered by 🤗 transformers and open-source datasets.
https://huggingface.co/dfurman
Apache License 2.0
64 stars 8 forks source link

Roadmap to include SFT + Alignment #2

Closed vikram71198 closed 1 year ago

vikram71198 commented 1 year ago

Hi @daniel-furman , I want to start off by saying this is a really cool repo! These scripts are extremely useful to a novice starting off with these libraries.

I mostly just see SFT notebooks. Do you have any plans to add in alignment as well (RLHF, RLAIF, PPO, DPO) after SFT-ing your models? Something along the lines of what HuggingFace has here.

If you're open to contributors, I'd love to join too if you have any other ideas.

Cheers!

daniel-furman commented 1 year ago

I am planning on adding alignment (DPO is next) - yet, not open to contributors at this time. Thanks for your nice comments and interest!