[FEATURE] Training with QLoRA + FSDP

h2oai / h2o-llmstudio

H2O LLM Studio - a framework and no-code GUI for fine-tuning LLMs. Documentation: https://h2oai.github.io/h2o-llmstudio/

https://gpt-gm.h2o.ai

Apache License 2.0

3.75k stars 391 forks source link

Open pascal-pfeiffer opened 3 months ago

pascal-pfeiffer commented 3 months ago

New advancements bringing quantized LoRA and FSDP together. https://github.com/AnswerDotAI/fsdp_qlora

Train larger models on consumer GPUs or older generation Data Center GPUs such as V100 Lets you finetune Llama-2 70B on dual 24GB GPUs.

AZ777xx commented 2 weeks ago

Would be awesome! Looking forward to it.