search

microsoft / onnxruntime

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
https://onnxruntime.ai
MIT License
14.83k stars 2.94k forks source link

[Build] Error when trying to build docker image of ONNXRuntime with CUDA support #22945

Open banu-narayana-dnjp opened 1 day ago

banu-narayana-dnjp commented 1 day ago

Describe the issue

While running the script, it expects to find "tools/ci_build/github/linux/docker/inference/x86_64/python/cpu/scripts/requirements.txt". But, there is no such file in that path.

Urgency

No response

Target platform

CUDA

Build script

git submodule update --init docker build -t onnxruntime-cuda --build-arg CUDA_VERSION=12.6.1 \ --build-arg CUDNN_VERSION=9.5.0.50 \ --build-arg GIT_BRANCH=$(git rev-parse --abbrev-ref HEAD) \ --build-arg GIT_COMMIT=$(git rev-parse HEAD) \ --build-arg ONNXRUNTIME_VERSION=$(cat ../VERSION_NUMBER) \ -f Dockerfile.cuda ..

Error / output

=> ERROR [stage-0 5/5] RUN cd /code && python3 -m venv /code/env && . /code/env/bin/activate && pip ins 3.9s

[stage-0 5/5] RUN cd /code && python3 -m venv /code/env && . /code/env/bin/activate && pip install --upgrade psutil setuptools wheel packaging && pip install -r tools/ci_build/github/linux/docker/inference/x86_64/python/cpu/scripts/requirements.txt && python /code/tools/ci_build/build.py --build_dir /code/build/Linux --allow_running_as_root --skip_submodule_sync --use_cuda --cuda_home /usr/local/cuda --cudnn_home /code/build/cudnn --build_shared_lib --skip_tests --config Release --build_wheel --update --build --parallel --cmake_generator Ninja --cmake_extra_defines ONNXRUNTIME_VERSION=$(cat ./VERSION_NUMBER) "CMAKE_CUDA_ARCHITECTURES=61;70;75;80;86;90" onnxruntime_BUILD_UNIT_TESTS=OFF: 2.365 Collecting psutil 2.531 Downloading psutil-6.1.0-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (22 kB) 2.689 Collecting setuptools 2.706 Downloading setuptools-75.6.0-py3-none-any.whl.metadata (6.7 kB) 2.763 Collecting wheel 2.781 Downloading wheel-0.45.1-py3-none-any.whl.metadata (2.3 kB) 2.825 Collecting packaging 2.843 Downloading packaging-24.2-py3-none-any.whl.metadata (3.2 kB) 2.897 Downloading psutil-6.1.0-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (287 kB) 2.934 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 287.3/287.3 kB 8.1 MB/s eta 0:00:00 2.953 Downloading setuptools-75.6.0-py3-none-any.whl (1.2 MB) 3.040 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 14.7 MB/s eta 0:00:00 3.058 Downloading wheel-0.45.1-py3-none-any.whl (72 kB) 3.068 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 72.5/72.5 kB 10.4 MB/s eta 0:00:00 3.087 Downloading packaging-24.2-py3-none-any.whl (65 kB) 3.094 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 65.5/65.5 kB 13.3 MB/s eta 0:00:00 3.146 Installing collected packages: wheel, setuptools, psutil, packaging 3.673 Successfully installed packaging-24.2 psutil-6.1.0 setuptools-75.6.0 wheel-0.45.1 3.866 ERROR: Could not open requirements file: [Errno 2] No such file or directory: 'tools/ci_build/github/linux/docker/inference/x86_64/python/cpu/scripts/requirements.txt'

Dockerfile.cuda:47

46 | # Create a virtual environment and install dependencies, then build ONNX Runtime with CUDA support. 47 | >>> RUN cd /code \ 48 | >>> && python3 -m venv /code/env \ 49 | >>> && . /code/env/bin/activate \ 50 | >>> && pip install --upgrade psutil setuptools wheel packaging \ 51 | >>> && pip install -r tools/ci_build/github/linux/docker/inference/x86_64/python/cpu/scripts/requirements.txt \ 52 | >>> # && pip install -r /home/banu_narayana/Documents/onnxruntime/tools/ci_build/github/linux/docker/inference/x86_64/python/cpu/scripts/requirements.txt \ 53 | >>> && python /code/tools/ci_build/build.py --build_dir /code/build/Linux \ 54 | >>> --allow_running_as_root --skip_submodule_sync \ 55 | >>> --use_cuda --cuda_home /usr/local/cuda \ 56 | >>> --cudnn_home /code/build/cudnn \ 57 | >>> --build_shared_lib --skip_tests \ 58 | >>> --config Release --build_wheel --update --build --parallel \ 59 | >>> --cmake_generator Ninja \ 60 | >>> --cmake_extra_defines ONNXRUNTIME_VERSION=$(cat ./VERSION_NUMBER) "CMAKE_CUDA_ARCHITECTURES=${CMAKE_CUDA_ARCHITECTURES}" onnxruntime_BUILD_UNIT_TESTS=OFF 61 |

ERROR: failed to solve: process "/bin/sh -c cd /code && python3 -m venv /code/env && . /code/env/bin/activate && pip install --upgrade psutil setuptools wheel packaging && pip install -r tools/ci_build/github/linux/docker/inference/x86_64/python/cpu/scripts/requirements.txt && python /code/tools/ci_build/build.py --build_dir /code/build/Linux --allow_running_as_root --skip_submodule_sync --use_cuda --cuda_home /usr/local/cuda --cudnn_home /code/build/cudnn --build_shared_lib --skip_tests --config Release --build_wheel --update --build --parallel --cmake_generator Ninja --cmake_extra_defines ONNXRUNTIME_VERSION=$(cat ./VERSION_NUMBER) \"CMAKE_CUDA_ARCHITECTURES=${CMAKE_CUDA_ARCHITECTURES}\" onnxruntime_BUILD_UNIT_TESTS=OFF" did not complete successfully: exit code: 1

Visual Studio Version

No response

GCC / Compiler Version

No response

fs-eire commented 1 day ago

@banu-narayana-dnjp could you help to verify if #22946 fixes this issue?

banu-narayana-dnjp commented 1 day ago

@banu-narayana-dnjp could you help to verify if #22946 fixes this issue?

No, the same error persists even if there is a requirements.txt file.

=> ERROR [stage-0 5/5] RUN cd /code && python3 -m venv /code/env && . /code/env/bin/activate && pip install --upgrad 3.9s

[stage-0 5/5] RUN cd /code && python3 -m venv /code/env && . /code/env/bin/activate && pip install --upgrade psutil setuptools wheel packaging && pip install -r tools/ci_build/github/linux/python/requirements.txt && python /code/tools/ci_build/build.py --build_dir /code/build/Linux --allow_running_as_root --skip_submodule_sync --use_cuda --cuda_home /usr/local/cuda --cudnn_home /code/build/cudnn --build_shared_lib --skip_tests --config Release --build_wheel --update --build --parallel --cmake_generator Ninja --cmake_extra_defines onnxruntime_ENABLE_NVTX_PROFILE=ON --cmake_extra_defines ONNXRUNTIME_VERSION=$(cat ./VERSION_NUMBER) "CMAKE_CUDA_ARCHITECTURES=61;70;75;80;86;90" onnxruntime_BUILD_UNIT_TESTS=OFF: 2.238 Collecting psutil 2.404 Downloading psutil-6.1.0-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (22 kB) 2.560 Collecting setuptools 2.577 Downloading setuptools-75.6.0-py3-none-any.whl.metadata (6.7 kB) 2.624 Collecting wheel 2.641 Downloading wheel-0.45.1-py3-none-any.whl.metadata (2.3 kB) 2.686 Collecting packaging 2.707 Downloading packaging-24.2-py3-none-any.whl.metadata (3.2 kB) 2.763 Downloading psutil-6.1.0-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (287 kB) 2.800 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 287.3/287.3 kB 8.1 MB/s eta 0:00:00 2.820 Downloading setuptools-75.6.0-py3-none-any.whl (1.2 MB) 2.990 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 7.3 MB/s eta 0:00:00 3.009 Downloading wheel-0.45.1-py3-none-any.whl (72 kB) 3.017 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 72.5/72.5 kB 15.3 MB/s eta 0:00:00 3.036 Downloading packaging-24.2-py3-none-any.whl (65 kB) 3.043 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 65.5/65.5 kB 14.4 MB/s eta 0:00:00 3.095 Installing collected packages: wheel, setuptools, psutil, packaging 3.630 Successfully installed packaging-24.2 psutil-6.1.0 setuptools-75.6.0 wheel-0.45.1 3.825 ERROR: Could not open requirements file: [Errno 2] No such file or directory: 'tools/ci_build/github/linux/python/requirements.txt'

Dockerfile.cuda:47

46 | # Create a virtual environment and install dependencies, then build ONNX Runtime with CUDA support. 47 | >>> RUN cd /code \ 48 | >>> && python3 -m venv /code/env \ 49 | >>> && . /code/env/bin/activate \ 50 | >>> && pip install --upgrade psutil setuptools wheel packaging \ 51 | >>> # && pip install -r tools/ci_build/github/linux/docker/inference/x86_64/python/cpu/scripts/requirements.txt \ 52 | >>> && pip install -r tools/ci_build/github/linux/python/requirements.txt \ 53 | >>> && python /code/tools/ci_build/build.py --build_dir /code/build/Linux \ 54 | >>> --allow_running_as_root --skip_submodule_sync \ 55 | >>> --use_cuda --cuda_home /usr/local/cuda \ 56 | >>> --cudnn_home /code/build/cudnn \ 57 | >>> --build_shared_lib --skip_tests \ 58 | >>> --config Release --build_wheel --update --build --parallel \ 59 | >>> --cmake_generator Ninja \ 60 | >>> --cmake_extra_defines onnxruntime_ENABLE_NVTX_PROFILE=ON \ 61 | >>> --cmake_extra_defines ONNXRUNTIME_VERSION=$(cat ./VERSION_NUMBER) "CMAKE_CUDA_ARCHITECTURES=${CMAKE_CUDA_ARCHITECTURES}" onnxruntime_BUILD_UNIT_TESTS=OFF 62 |

ERROR: failed to solve: process "/bin/sh -c cd /code && python3 -m venv /code/env && . /code/env/bin/activate && pip install --upgrade psutil setuptools wheel packaging && pip install -r tools/ci_build/github/linux/python/requirements.txt && python /code/tools/ci_build/build.py --build_dir /code/build/Linux --allow_running_as_root --skip_submodule_sync --use_cuda --cuda_home /usr/local/cuda --cudnn_home /code/build/cudnn --build_shared_lib --skip_tests --config Release --build_wheel --update --build --parallel --cmake_generator Ninja --cmake_extra_defines onnxruntime_ENABLE_NVTX_PROFILE=ON --cmake_extra_defines ONNXRUNTIME_VERSION=$(cat ./VERSION_NUMBER) \"CMAKE_CUDA_ARCHITECTURES=${CMAKE_CUDA_ARCHITECTURES}\" onnxruntime_BUILD_UNIT_TESTS=OFF" did not complete successfully: exit code: 1

fs-eire commented 20 hours ago

3.825 ERROR: Could not open requirements file: [Errno 2] No such file or directory: 'tools/ci_build/github/linux/python/requirements.txt'

This file should exist.

could you try update

pip install -r tools/ci_build/github/linux/python/requirements.txt

to 

pip install -r /code/tools/ci_build/github/linux/python/requirements.txt