search

sasha0552 / vllm-ci

CI scripts designed to build a Pascal-compatible version of vLLM.
MIT License
8 stars 1 forks source link

'manylinux2014_x86_64' causing trouble with triton wheel on Ubuntu #1

Closed the-crypt-keeper closed 1 month ago

the-crypt-keeper commented 1 month ago

Hi @sasha0552! First wanted to say thanks for the work you're doing here, I ran into the "Cannot convert f16 to f16" issue and saw the triton folks turned down your PR - not very cool of them to drop everything before Ampere imo, that turns a ton of perfectly good hardware into ewaste.

My actual issue here is that on Ubuntu22 I am unable to install your triton wheel

ERROR: triton-2.3.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl is not a supported wheel on this platform.

Is there a chance we could get this one compiled for normal linux_x86_64 like the vllm one is?

Many thanks!

sasha0552 commented 1 month ago

Hi @the-crypt-keeper.

It looks like your version of python is older/newer than 3.11. Can you run python3 --version and send it to me? Theoretically, I could include a build for 3.8-3.11 (these versions are supported by vllm) and triton for 3.7-3.12 (these versions are supported by triton).

sasha0552 commented 1 month ago

I have included a build for all supported versions of python. Check the v2 release. I plan to create a python repository later to make installation easier (pip3 --extra-index-url ... install vllm, which will install the correct vllm and triton)

sasha0552 commented 1 month ago

Check out the new documentation in the README.md.

the-crypt-keeper commented 1 month ago

@sasha0552 It works perfectly! I cannot thank you enough ❤️