feat: any plan to support NPU

bentoml / OpenLLM

Run any open-source LLMs, such as Llama 3.1, Gemma, as OpenAI compatible API endpoint in the cloud.

https://bentoml.com

Apache License 2.0

9.7k stars 616 forks source link

Open zhangxinyang97 opened 4 months ago

zhangxinyang97 commented 4 months ago

support Ascend NPU

more and more developer using NPU, and vllm is also interesting in NPU ecosystem, https://github.com/vllm-project/vllm/issues/1606.

No response

aarnphm commented 3 months ago

we don't have any immediate plan atm.