search

AnswerDotAI / fsdp_qlora

Training LLMs with QLoRA + FSDP
Apache License 2.0
1.42k stars 188 forks source link

Bigger context size? #38

Open LoganALJones opened 7 months ago

LoganALJones commented 7 months ago

Is training with 1024 or 2048 sequence length feasible using this method?