search

pytorch / torchtune

PyTorch native finetuning library
https://pytorch.org/torchtune/main/
BSD 3-Clause "New" or "Revised" License
4.24k stars 417 forks source link

[Config] Do we need to avoid combinatorial number of configs? #1282

Open felipemello1 opened 2 months ago

felipemello1 commented 2 months ago

Currently, for every llama SKU, we have 6 configs:

We do not have DPO/PPO (I guess we probably should), but this could easily add another 6 configs (QLoRA, LoRA, Full) * (distributed, single).

If/when we support DoRA, would we add another 8? (DoRA + QDoRA)(single, distributed) (sft, dpo)

Not sure if this needs a solution. Posting it for the discussion. I thought of a few ideas, but i don't think I like them very much 1) Make QLoRA (and other variants) a parameter in LoRA configs? 2) Make model size a parameter? --sku llama8b 3) Only keep single/distributed configs, and make PeFT an optional parameter. We could create a single peft.yaml with just the relevant parameters, shared by all models; 💩 4) Make distributed a parameter? 💩 5) Dont touch it

gau-nernst commented 2 months ago

I think merging QLoRA with LoRA make sense. After all, QLoRA = LoRA + NF4 weight. So the NF4 quant can be an option inside a LoRA recipe. Might help to address #1093 too.