search

hpcaitech / ColossalAI

Making large AI models cheaper, faster and more accessible
https://www.colossalai.org
Apache License 2.0
38.83k stars 4.35k forks source link

[FEATURE]: Lora/QLora in GeminiPlugin and TorchFSDP #6138

Open ericxsun opened 6 days ago

ericxsun commented 6 days ago

Describe the feature

How can we support LoRA/QLoRA in Gemini or TorchFSDP plugin?

If there’s documentation on this feature, it might encourage community contributions.

Thanks a lot.

ericxsun commented 4 days ago

Fond this PR https://github.com/hpcaitech/ColossalAI/pull/5001.

It’s been a long time since the last update.