Merging with vLLM main branch

EmbeddedLLM / vllm-rocm

vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs

https://vllm.readthedocs.io

Apache License 2.0

83 stars 5 forks source link

Merging with vLLM main branch #12

Closed zhuohan123 closed 6 months ago

zhuohan123 commented 7 months ago

Hi EmbeddedLLM team,

We are the maintainers of the vLLM project. We just found this project and it's very exciting! Are you interested in contributing the fork to the main branch to add official support to ROCM in vLLM? Feel free to reach out to me at zhuohan[at]berkeley.edu and happy to help in any way.

Thanks, Zhuohan

tanpinsiang commented 7 months ago

Absolutely, @zhuohan123. We're thrilled at the opportunity to contribute to the vLLM project and fully committed to integrating ROCm support. Thank you for reaching out and offering your help.

You can monitor our progress and collaboration on this PR: https://github.com/vllm-project/vllm/pull/1749.

tanpinsiang commented 6 months ago

The ROCm support has been successfully merged. You can find the changes in the following pull request: https://github.com/vllm-project/vllm/pull/1836.