search

sgl-project / sglang

SGLang is a fast serving framework for large language models and vision language models.
https://sglang.readthedocs.io/en/latest/
Apache License 2.0
5.42k stars 399 forks source link

Metal support? #23

Closed mitkox closed 2 months ago

mitkox commented 8 months ago

Hey, when is planned the support for Metal backend?

Koloth commented 8 months ago

I'm also interested in this, are their plans for this on the roadmap?

ChristianWeyer commented 6 months ago

Highly appreciated to have Metal support!

github-actions[bot] commented 2 months ago

This issue has been automatically closed due to inactivity. Please feel free to reopen it if needed.

digisomni commented 1 month ago

Would be nice as I want to deploy this at enterprise scale but need to quickly test implementation locally.