[Feature] Jamba 1.5 Support PLS - Githubissues

sgl-project / sglang

SGLang is a fast serving framework for large language models and vision language models.

https://sglang.readthedocs.io/en/latest/

Apache License 2.0

5.16k stars 364 forks source link

[Feature] Jamba 1.5 Support PLS #1190

Open nivibilla opened 3 weeks ago

nivibilla commented 3 weeks ago

Checklist

[x] 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
[x] 2. Please use English, otherwise it will be closed.

Motivation

First SOTA ssm based model, vllm currently supports it but there is some parallel work in vllm to optimise it aswell

https://huggingface.co/collections/ai21labs/jamba-15-66c44befa474a917fcf55251

Related resources

vllm implementation https://github.com/vllm-project/vllm/pull/4115

merrymercy commented 3 weeks ago

We would love to have this model. Contributions are welcome!