[Feature]: support Ascend 910B in the future

vllm-project / vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

https://docs.vllm.ai

Apache License 2.0

22.26k stars 3.14k forks source link

[Feature]: support Ascend 910B in the future #6066

Open jkl375 opened 3 days ago

jkl375 commented 3 days ago

🚀 The feature, motivation and pitch

Do you have plans to support Ascend 910B in the future？

Alternatives

No response

Additional context

No response

mgoin commented 3 days ago

Hi @jkl375 we very much accept contributions for new hardware targets! Could you share any resources in terms of Ascend's support for PyTorch to make it easier to scope for vLLM?

jkl375 commented 2 days ago

OK, Do you have any hardware resources for Ascend 910b? https://www.hiascend.com/developer/ascendhub/detail/bcf86497fd874004b3b0a6e5d533081e this is the docker image for ascend llm models, witch have torch_npu. https://gitee.com/ascend/pytorch This repo is ascend Extension for PyTorch.