llama 3 -support? - Githubissues

artidoro / qlora

QLoRA: Efficient Finetuning of Quantized LLMs

https://arxiv.org/abs/2305.14314

MIT License

9.96k stars 820 forks source link

llama 3 -support? #296

Open LuoyaoChen opened 3 months ago

LuoyaoChen commented 3 months ago

Hi,

Has anyone tried LLAMA-3 using this codebase? Mine is not working with llama3-8b. i.e. it reports no errors, and training was able to start. However, it got stuck at step1. Not sure whether I should expect this code to support llama3-8b to start with.

Anyone has any experience, I would like to hear more!

Thanks!