[Feature] Request for Support for Ascend Series Graphics Cards

[junior-zsy]

Motivation

Hello,

I hope this message finds you well. I am writing to kindly request your support for the Ascend series of graphics cards in your project.

As you may be aware, recent geopolitical circumstances have resulted in the inability for developers in China to access NVIDIA graphics cards due to export restrictions from the United States. As a result, many of us have turned to domestically produced alternatives, such as the Ascend series graphics cards, which have proven to be highly capable and efficient.

However, the lack of support for the Ascend series in many projects, including this one, has presented a significant challenge. It would be greatly appreciated if you could consider extending your support to include these graphics cards, thereby enabling us to continue contributing to and benefiting from your project.

I understand that this may require additional resources and effort on your part. To assist you in this endeavor, I have included the link to the official Ascend development resources on GitHub and their official website:

GitHub: https://github.com/Ascend Official Website: https://www.hiascend.com/en/ These resources provide comprehensive documentation and development tools that should hopefully make the process of integrating support for the Ascend series more straightforward.

I sincerely appreciate your consideration of this request. The support for Ascend series graphics cards would not only benefit developers in China, but also those across the globe who may choose or need to use these cards.

Thank you for your time and understanding. I look forward to your positive response.

Related resources

GitHub: https://github.com/Ascend Official Website: https://www.hiascend.com/en/

Additional context

No response

[lvhan028]

LMDeploy has no plan to support other devices than NVIDIA this year.

[datalee]

LMDeploy has no plan to support other devices than NVIDIA this year.

这个今年会考虑吗？这个部署框架确实很厉害，尤其在长上下文场景下对显存的控制，是否有适配昇腾的计划？期待

[lvhan028]

很遗憾，目前，还没有ascend芯片支持的计划

[datalee]

很遗憾，目前，还没有ascend芯片支持的计划

好的，明白，感谢

[eigen2017]

我想华为愿意提供免费资源。 N卡已经禁售了，fastllm已经开始兼容昇腾了。 国内的推理加速框架就看lmdeploy，还有fastllm了~~ 加油~~

[eigen2017]

@datalee 另外昇腾有自己的MindIE推理加速框架。可以试试。

[lvhan028]

与deeplink团队合作，在最新的main分支，已经支持了910b。

[eigen2017]

万岁

与deeplink团队合作，在最新的main分支，已经支持了910b。

[datalee]

与deeplink团队合作，在最新的main分支，已经支持了910b。

有链接吗？大佬，怎么使用

[jeejeelee]

与deeplink团队合作，在最新的main分支，已经支持了910b。

太棒了，请问如何使用呢，目前看依赖deeplink_ext这个模块,如何下载或者编译这个模块呢，如果有文档的话，将会非常完美~

[lvhan028]

我们尽快提供文档和镜像

[tangpanyu]

我们尽快提供文档和镜像

你好我在安装时遇到：

CMake Error at /root/miniconda3/envs/dipu/lib/python3.9/site-packages/cmake/data/share/cmake-3.30/Modules/Internal/CMakeCUDAFindToolkit.cmake:104 (message):
  Failed to find nvcc.

  Compiler requires the CUDA toolkit.  Please set the CUDAToolkit_ROOT
  variable.
Call Stack (most recent call first):
  /root/miniconda3/envs/dipu/lib/python3.9/site-packages/cmake/data/share/cmake-3.30/Modules/CMakeDetermineCUDACompiler.cmake:85 (cmake_cuda_find_toolkit)
  CMakeLists.txt:16 (project)

我是910B，为啥需要CUDA呢？现在有910B的文档吗？

[lvhan028]

在910B上，是使用 pytorch engine推理的。不能使用 turbomind engine。所以，不要编译 turbomind 的源码。 在 pytorch engine上，LMDeploy 与 Deeplink 团队合作，完成对 910b 的支持，需要安装 deeplink 相关的依赖。但是，目前deeplink并没有提供 pypi 包。推荐还是镜像的方式。 只是，deeplink使用的镜像包含了huawei 商用 cann，不确定是不是可以公开，目前还在协调过程中。 有进展后，我们会及时通知的。

[viningz]

请问后续会加入对天数系列加速卡的支持吗？

[lvhan028]

能不能引荐下天数的朋友？可以聊聊看