search

Lightning-AI / litgpt

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
https://lightning.ai
Apache License 2.0
9.49k stars 944 forks source link

Qwen support #840

Open vabatista opened 8 months ago

vabatista commented 8 months ago

It would be very helpful if lit-gpt provides support for the new models Qwen (https://huggingface.co/Qwen). They have strong performance compared to other open source models.

carmocca commented 8 months ago

After a brief skim over the HF implementation, I don't see any blockers to support it. Contributions are welcome!

vabatista commented 5 months ago

Maybe you can also add support for Qwen 1.5.