search

meta-llama / llama-recipes

Scripts for fine-tuning Meta Llama3 with composable FSDP & PEFT methods to cover single/multi-node GPUs. Supports default & custom datasets for applications such as summarization and Q&A. Supporting a number of candid inference solutions such as HF TGI, VLLM for local or cloud deployment. Demo apps to showcase Meta Llama3 for WhatsApp & Messenger.
9.95k stars 1.4k forks source link

Finetuning general LLM models from hugging face #521

Closed bkhanal-11 closed 1 week ago

bkhanal-11 commented 2 weeks ago

Thank you for your continuous support to the LLM open-source community.

I was wondering if we use AutoModel, AutoConfig and AutoTokenizer instead of LlamaForCausalLM, LlamaConfig and LlamaTokenizer for general LLM fine-tuning, will the fine-tuning pipeline/recipe fail?

mreso commented 1 week ago

Hi @bkhanal-11 for the single gpu use case this might work, but for the advances use cases with FSDP you'll need to different wrapping policy and there might be other things to consider and adapt.

bkhanal-11 commented 1 week ago

That makes sense, thanks.