search

runpod-workers / worker-vllm

The RunPod worker template for serving our large language model endpoints. Powered by vLLM.
MIT License
220 stars 85 forks source link

Build not possilbe #26

Closed saaldigital-cw closed 8 months ago

saaldigital-cw commented 8 months ago

I tried to build the docker from scratch but also get an error (using CUDA 11.8, runpod/base:0.4.4)

RUN python3.11 -m pip install -e git+https://github.com/runpod/vllm-fork-for-sls-worker.git@cuda-11.8#egg=vllm 150.2s

[builder 5/7] RUN python3.11 -m pip install -e git+https://github.com/runpod/vllm-fork-for-sls-worker.git@cuda-11.8#egg=vllm: 0.734 Obtaining vllm from git+https://github.com/runpod/vllm-fork-for-sls-worker.git@cuda-11.8#egg=vllm 0.734 Cloning https://github.com/runpod/vllm-fork-for-sls-worker.git (to revision cuda-11.8) to /src/vllm 0.741 Running command git clone --filter=blob:none --quiet https://github.com/runpod/vllm-fork-for-sls-worker.git /src/vllm 1.802 Running command git checkout -b cuda-11.8 --track origin/cuda-11.8 2.117 Switched to a new branch 'cuda-11.8' 2.117 Branch 'cuda-11.8' set up to track remote branch 'cuda-11.8' from 'origin'. 2.118 Resolved https://github.com/runpod/vllm-fork-for-sls-worker.git to commit 1de0211a7c2b0f736f1afd21f02740ec542c2e55 2.118 Preparing metadata (setup.py): started 3.241 Preparing metadata (setup.py): finished with status 'done' 3.378 Collecting ninja (from vllm) 3.703 Downloading ninja-1.11.1.1-py2.py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl.metadata (5.3 kB) 3.775 Collecting psutil (from vllm) 3.870 Downloading psutil-5.9.7-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (21 kB) 3.953 Collecting ray>=2.5.1 (from vllm) 4.048 Downloading ray-2.8.1-cp311-cp311-manylinux2014_x86_64.whl.metadata (13 kB) 4.141 Collecting pandas (from vllm) 4.236 Downloading pandas-2.1.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (18 kB) 4.300 Collecting pyarrow (from vllm) 4.395 Downloading pyarrow-14.0.2-cp311-cp311-manylinux_2_28_x86_64.whl.metadata (3.0 kB) 4.436 Collecting sentencepiece (from vllm) 4.535 Downloading sentencepiece-0.1.99-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB) 4.978 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 3.0 MB/s eta 0:00:00 4.980 Requirement already satisfied: numpy in /usr/local/lib/python3.11/dist-packages (from vllm) (1.24.1) 4.981 Requirement already satisfied: packaging in /usr/local/lib/python3.11/dist-packages (from vllm) (23.2) 5.013 Collecting transformers>=4.36.0 (from vllm) 5.107 Downloading transformers-4.36.2-py3-none-any.whl.metadata (126 kB) 5.114 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 126.8/126.8 kB 22.7 MB/s eta 0:00:00 5.121 Requirement already satisfied: fastapi in /usr/local/lib/python3.11/dist-packages (from vllm) (0.105.0) 5.122 Requirement already satisfied: uvicorn[standard] in /usr/local/lib/python3.11/dist-packages (from vllm) (0.24.0.post1) 5.190 Collecting pydantic==1.10.13 (from vllm) 5.285 Downloading pydantic-1.10.13-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (149 kB) 5.293 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 149.6/149.6 kB 20.8 MB/s eta 0:00:00 5.419 Collecting aioprometheus[starlette] (from vllm) 5.516 Downloading aioprometheus-23.3.0-py3-none-any.whl (31 kB) 5.522 Requirement already satisfied: typing-extensions>=4.2.0 in /usr/local/lib/python3.11/dist-packages (from pydantic==1.10.13->vllm) (4.9.0) 5.590 Requirement already satisfied: click>=7.0 in /usr/local/lib/python3.11/dist-packages (from ray>=2.5.1->vllm) (8.1.7) 5.590 Requirement already satisfied: filelock in /usr/local/lib/python3.11/dist-packages (from ray>=2.5.1->vllm) (3.9.0) 5.617 Collecting jsonschema (from ray>=2.5.1->vllm) 5.711 Downloading jsonschema-4.20.0-py3-none-any.whl.metadata (8.1 kB) 5.751 Collecting msgpack<2.0.0,>=1.0.0 (from ray>=2.5.1->vllm) 5.846 Downloading msgpack-1.0.7-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (9.1 kB) 5.950 Collecting protobuf!=3.19.5,>=3.15.3 (from ray>=2.5.1->vllm) 6.047 Downloading protobuf-4.25.1-cp37-abi3-manylinux2014_x86_64.whl.metadata (541 bytes) 6.051 Requirement already satisfied: pyyaml in /usr/local/lib/python3.11/dist-packages (from ray>=2.5.1->vllm) (6.0.1) 6.051 Requirement already satisfied: aiosignal in /usr/local/lib/python3.11/dist-packages (from ray>=2.5.1->vllm) (1.3.1) 6.052 Requirement already satisfied: frozenlist in /usr/local/lib/python3.11/dist-packages (from ray>=2.5.1->vllm) (1.4.1) 6.053 Requirement already satisfied: requests in /usr/local/lib/python3.11/dist-packages (from ray>=2.5.1->vllm) (2.31.0) 6.182 Requirement already satisfied: huggingface-hub<1.0,>=0.19.3 in /usr/local/lib/python3.11/dist-packages (from transformers>=4.36.0->vllm) (0.20.1) 6.389 Collecting regex!=2019.12.17 (from transformers>=4.36.0->vllm) 6.484 Downloading regex-2023.10.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (40 kB) 6.490 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.9/40.9 kB 6.1 MB/s eta 0:00:00 6.580 Collecting tokenizers<0.19,>=0.14 (from transformers>=4.36.0->vllm) 6.674 Downloading tokenizers-0.15.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.7 kB) 6.731 Collecting safetensors>=0.3.1 (from transformers>=4.36.0->vllm) 6.831 Downloading safetensors-0.4.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB) 6.837 Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.11/dist-packages (from transformers>=4.36.0->vllm) (4.66.1) 6.843 Requirement already satisfied: orjson in /usr/local/lib/python3.11/dist-packages (from aioprometheus[starlette]->vllm) (3.9.10) 6.966 Collecting quantile-python>=1.1 (from aioprometheus[starlette]->vllm) 7.061 Downloading quantile-python-1.1.tar.gz (2.9 kB) 7.066 Preparing metadata (setup.py): started 7.147 Preparing metadata (setup.py): finished with status 'done' 7.149 Requirement already satisfied: starlette>=0.14.2 in /usr/local/lib/python3.11/dist-packages (from aioprometheus[starlette]->vllm) (0.27.0) 7.156 Requirement already satisfied: anyio<4.0.0,>=3.7.1 in /usr/local/lib/python3.11/dist-packages (from fastapi->vllm) (3.7.1) 7.227 Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.11/dist-packages (from pandas->vllm) (2.8.2) 7.266 Collecting pytz>=2020.1 (from pandas->vllm) 7.361 Downloading pytz-2023.3.post1-py2.py3-none-any.whl.metadata (22 kB) 7.383 Collecting tzdata>=2022.1 (from pandas->vllm) 7.477 Downloading tzdata-2023.3-py2.py3-none-any.whl (341 kB) 7.491 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 341.8/341.8 kB 25.8 MB/s eta 0:00:00 7.512 Requirement already satisfied: h11>=0.8 in /usr/local/lib/python3.11/dist-packages (from uvicorn[standard]->vllm) (0.14.0) 7.513 Requirement already satisfied: httptools>=0.5.0 in /usr/local/lib/python3.11/dist-packages (from uvicorn[standard]->vllm) (0.6.1) 7.514 Requirement already satisfied: python-dotenv>=0.13 in /usr/local/lib/python3.11/dist-packages (from uvicorn[standard]->vllm) (1.0.0) 7.516 Requirement already satisfied: uvloop!=0.15.0,!=0.15.1,>=0.14.0 in /usr/local/lib/python3.11/dist-packages (from uvicorn[standard]->vllm) (0.19.0)
7.517 Requirement already satisfied: watchfiles>=0.13 in /usr/local/lib/python3.11/dist-packages (from uvicorn[standard]->vllm) (0.21.0) 7.518 Requirement already satisfied: websockets>=10.4 in /usr/local/lib/python3.11/dist-packages (from uvicorn[standard]->vllm) (12.0) 7.527 Requirement already satisfied: idna>=2.8 in /usr/local/lib/python3.11/dist-packages (from anyio<4.0.0,>=3.7.1->fastapi->vllm) (3.6) 7.527 Requirement already satisfied: sniffio>=1.1 in /usr/local/lib/python3.11/dist-packages (from anyio<4.0.0,>=3.7.1->fastapi->vllm) (1.3.0) 7.568 Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.11/dist-packages (from huggingface-hub<1.0,>=0.19.3->transformers>=4.36.0->vllm) (2023.12.2) 7.576 Requirement already satisfied: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil>=2.8.2->pandas->vllm) (1.16.0) 7.656 Requirement already satisfied: attrs>=22.2.0 in /usr/local/lib/python3.11/dist-packages (from jsonschema->ray>=2.5.1->vllm) (23.1.0) 7.673 Collecting jsonschema-specifications>=2023.03.6 (from jsonschema->ray>=2.5.1->vllm) 7.769 Downloading jsonschema_specifications-2023.11.2-py3-none-any.whl.metadata (3.0 kB) 7.798 Collecting referencing>=0.28.4 (from jsonschema->ray>=2.5.1->vllm) 7.892 Downloading referencing-0.32.0-py3-none-any.whl.metadata (2.7 kB) 8.007 Collecting rpds-py>=0.7.1 (from jsonschema->ray>=2.5.1->vllm) 8.102 Downloading rpds_py-0.15.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.1 kB) 8.116 Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.11/dist-packages (from requests->ray>=2.5.1->vllm) (3.3.2) 8.118 Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.11/dist-packages (from requests->ray>=2.5.1->vllm) (2.0.7) 8.118 Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.11/dist-packages (from requests->ray>=2.5.1->vllm) (2023.11.17) 8.292 Downloading pydantic-1.10.13-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB) 8.441 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.1/3.1 MB 20.7 MB/s eta 0:00:00 8.539 Downloading ray-2.8.1-cp311-cp311-manylinux2014_x86_64.whl (63.1 MB) 12.06 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 63.1/63.1 MB 11.6 MB/s eta 0:00:00 12.16 Downloading transformers-4.36.2-py3-none-any.whl (8.2 MB) 12.25 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.2/8.2 MB 100.0 MB/s eta 0:00:00 12.34 Downloading ninja-1.11.1.1-py2.py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl (307 kB) 12.35 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 307.2/307.2 kB 70.1 MB/s eta 0:00:00 12.45 Downloading pandas-2.1.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.2 MB) 13.01 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.2/12.2 MB 18.7 MB/s eta 0:00:00 13.11 Downloading psutil-5.9.7-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (285 kB) 13.12 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 285.5/285.5 kB 34.1 MB/s eta 0:00:00 13.21 Downloading pyarrow-14.0.2-cp311-cp311-manylinux_2_28_x86_64.whl (38.0 MB) 14.05 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 38.0/38.0 MB 43.3 MB/s eta 0:00:00 14.14 Downloading msgpack-1.0.7-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (557 kB) 14.16 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 558.0/558.0 kB 54.1 MB/s eta 0:00:00 14.25 Downloading protobuf-4.25.1-cp37-abi3-manylinux2014_x86_64.whl (294 kB) 14.26 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 294.6/294.6 kB 49.4 MB/s eta 0:00:00 14.35 Downloading pytz-2023.3.post1-py2.py3-none-any.whl (502 kB) 14.36 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 502.5/502.5 kB 54.5 MB/s eta 0:00:00 14.46 Downloading regex-2023.10.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (785 kB) 14.48 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 785.1/785.1 kB 52.0 MB/s eta 0:00:00 14.57 Downloading safetensors-0.4.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB) 14.60 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 53.5 MB/s eta 0:00:00 14.70 Downloading tokenizers-0.15.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.8 MB) 14.77 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.8/3.8 MB 57.7 MB/s eta 0:00:00 14.86 Downloading jsonschema-4.20.0-py3-none-any.whl (84 kB) 14.86 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 84.7/84.7 kB 25.3 MB/s eta 0:00:00 14.96 Downloading jsonschema_specifications-2023.11.2-py3-none-any.whl (17 kB) 15.06 Downloading referencing-0.32.0-py3-none-any.whl (26 kB) 15.16 Downloading rpds_py-0.15.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB) 15.18 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 56.1 MB/s eta 0:00:00 15.29 Building wheels for collected packages: quantile-python 15.29 Building wheel for quantile-python (setup.py): started 15.42 Building wheel for quantile-python (setup.py): finished with status 'done' 15.42 Created wheel for quantile-python: filename=quantile_python-1.1-py3-none-any.whl size=3444 sha256=69cee18b27427564f5b3fe833b047425f0cbdd09c7b26201b727b749af7bbf68 15.42 Stored in directory: /runpod-volume/.cache/pip/wheels/67/a2/17/29e7169adf03a7e44b922abb6a42c2c1b0fda11f7bfbdb24a2 15.42 Successfully built quantile-python 16.00 Installing collected packages: sentencepiece, quantile-python, pytz, ninja, tzdata, safetensors, rpds-py, regex, pydantic, pyarrow, psutil, protobuf, msgpack, aioprometheus, referencing, pandas, tokenizers, jsonschema-specifications, transformers, jsonschema, ray, vllm 16.21 Attempting uninstall: pydantic 16.21 Found existing installation: pydantic 2.5.2 16.22 Uninstalling pydantic-2.5.2: 16.23 Successfully uninstalled pydantic-2.5.2 23.25 Running setup.py develop for vllm 147.9 error: subprocess-exited-with-error 147.9 147.9 × python setup.py develop did not run successfully. 147.9 │ exit code: 1 147.9 ╰─> [221 lines of output] 147.9 No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda' 147.9 running develop

antonioglass commented 8 months ago

probably related to my issue: https://github.com/runpod-workers/worker-vllm/issues/25

alpayariyak commented 8 months ago

Moving this to #25 as it seems to also be related to building the image on a machine with no GPUs.