search

pytorch / audio

Data manipulation and transformation for audio signal processing, powered by PyTorch
https://pytorch.org/audio
BSD 2-Clause "Simplified" License
2.43k stars 636 forks source link

Installation with pip git+ is broken #3752

Open gdagil opened 4 months ago

gdagil commented 4 months ago

🐛 Describe the bug

My steps

I have built torch-v2.2.0 on my computer and want to install torchvision and torchaudio

  1. install torchvision - working fine

    CXX=g++-11 CUDA_ROOT=/usr \
        ${CONDA_DIR}/bin/python -m pip install --no-cache --no-cache-dir \
        git+https://github.com/pytorch/vision@v0.17.0
    with logs
    You see the torch with 2.2.0a0+gitunknown version, because i downloaded torch from releases
    ``` Collecting git+https://github.com/pytorch/vision@v0.17.0 Cloning https://github.com/pytorch/vision (to revision v0.17.0) to /tmp/pip-req-build-cfnenazs Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision /tmp/pip-req-build-cfnenazs Running command git checkout -q b2383d44751bf85e58cfb9223bbf4e5961c09fa1 Resolved https://github.com/pytorch/vision to commit b2383d44751bf85e58cfb9223bbf4e5961c09fa1 Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Requirement already satisfied: numpy in /home/user/conda/lib/python3.11/site-packages (from torchvision==0.17.0+b2383d4) (1.26.0) Requirement already satisfied: requests in /home/user/conda/lib/python3.11/site-packages (from torchvision==0.17.0+b2383d4) (2.31.0) Requirement already satisfied: torch in /home/user/conda/lib/python3.11/site-packages (from torchvision==0.17.0+b2383d4) (2.2.0a0+gitunknown) Collecting pillow!=8.3.*,>=5.3.0 (from torchvision==0.17.0+b2383d4) Downloading pillow-10.2.0-cp311-cp311-manylinux_2_28_x86_64.whl.metadata (9.7 kB) Requirement already satisfied: charset-normalizer<4,>=2 in /home/user/conda/lib/python3.11/site-packages (from requests->torchvision==0.17.0+b2383d4) (2.0.4) Requirement already satisfied: idna<4,>=2.5 in /home/user/conda/lib/python3.11/site-packages (from requests->torchvision==0.17.0+b2383d4) (3.4) Requirement already satisfied: urllib3<3,>=1.21.1 in /home/user/conda/lib/python3.11/site-packages (from requests->torchvision==0.17.0+b2383d4) (1.26.18) Requirement already satisfied: certifi>=2017.4.17 in /home/user/conda/lib/python3.11/site-packages (from requests->torchvision==0.17.0+b2383d4) (2024.2.2) Requirement already satisfied: filelock in /home/user/conda/lib/python3.11/site-packages (from torch->torchvision==0.17.0+b2383d4) (3.13.1) Requirement already satisfied: typing-extensions>=4.8.0 in /home/user/conda/lib/python3.11/site-packages (from torch->torchvision==0.17.0+b2383d4) (4.10.0) Requirement already satisfied: sympy in /home/user/conda/lib/python3.11/site-packages (from torch->torchvision==0.17.0+b2383d4) (1.12) Requirement already satisfied: networkx in /home/user/conda/lib/python3.11/site-packages (from torch->torchvision==0.17.0+b2383d4) (3.2.1) Requirement already satisfied: jinja2 in /home/user/conda/lib/python3.11/site-packages (from torch->torchvision==0.17.0+b2383d4) (3.1.3) Requirement already satisfied: fsspec in /home/user/conda/lib/python3.11/site-packages (from torch->torchvision==0.17.0+b2383d4) (2024.2.0) Requirement already satisfied: MarkupSafe>=2.0 in /home/user/conda/lib/python3.11/site-packages (from jinja2->torch->torchvision==0.17.0+b2383d4) (2.1.5) Requirement already satisfied: mpmath>=0.19 in /home/user/conda/lib/python3.11/site-packages (from sympy->torch->torchvision==0.17.0+b2383d4) (1.3.0) Downloading pillow-10.2.0-cp311-cp311-manylinux_2_28_x86_64.whl (4.5 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.5/4.5 MB 9.5 MB/s eta 0:00:00 Building wheels for collected packages: torchvision Building wheel for torchvision (pyproject.toml) ... done Created wheel for torchvision: filename=torchvision-0.17.0+b2383d4-cp311-cp311-linux_x86_64.whl size=988453 sha256=110b27c25384b026e0d8150bd223528eb70d7f587e5e5bb21544f33b7423e194 Stored in directory: /tmp/pip-ephem-wheel-cache-halmtbaq/wheels/dd/3b/e3/a0fb109cd18992fe8b57304416c5b8e968d2f1228db0959bfe Successfully built torchvision Installing collected packages: pillow, torchvision Successfully installed pillow-10.2.0 torchvision-0.17.0+b2383d4 ```

  2. install torchaudio - not working

    LD_LIBRARY_PATH="${CONDA_DIR}/lib:/usr/local/cuda/targets/x86_64-linux/lib${LD_LIBRARY_PATH:+${LD_LIBRARY_PATH}:}" \
CMAKE_FIND_FRAMEWORK=LAST CMAKE_SYSTEM_PROCESSOR=x86_64 \
GLIBCXX_USE_CXX11_ABI=1 \
TORCH_CUDA_ARCH_LIST="7.0;7.2;8.0;8.6;8.7;9.0" \
CMAKE_PREFIX_PATH=${CONDA_DIR:-"$(dirname $(which conda))/../"} \
CMAKE_INCLUDE_PATH="${CONDA_DIR}/include" \
CUDA_USE_STATIC_CUDA_RUNTIME=ON \
USE_SYSTEM_NCCL=ON USE_STATIC_NCCL=OFF NCCL_ROOT="/usr" NCCL_LIB_DIR="/usr/lib/x86_64-linux-gnu" NCCL_INCLUDE_DIR="/usr/include" CUDA_HOME="/usr/local/cuda" \
TORCH_VERSION="2.2.0a0+gitunknown" \
USE_CUSPARSELT=ON CUSPARSELT_INCLUDE_PATH="/usr/include/x86_64-linux-gnu" CUSPARSELT_LIBRARY_PATH="/usr/lib/x86_64-linux-gnu" \
USE_CUDA=ON USE_CUDNN=ON USE_STATIC_CUDNN=OFF CUDNN_ROOT="/usr" \
USE_FFMPEG=ON \
${CONDA_DIR}/bin/python -m pip install --no-cache --no-cache-dir \
        git+https://github.com/pytorch/audio@v2.2.0
    and have error ``` Collecting git+https://github.com/pytorch/audio@v2.2.0 Cloning https://github.com/pytorch/audio (to revision v2.2.0) to /tmp/pip-req-build-aa95t9du Running command git clone --filter=blob:none --quiet https://github.com/pytorch/audio /tmp/pip-req-build-aa95t9du Running command git checkout -q 08901ade5d17d3e3cf6fc039cbd601cbd2853686 Resolved https://github.com/pytorch/audio to commit 08901ade5d17d3e3cf6fc039cbd601cbd2853686 Running command git submodule update --init --recursive -q /bin/sh: /home/user/conda/lib/libtinfo.so.6: no version information available (required by /bin/sh) Installing build dependencies ... done Getting requirements to build wheel ... error error: subprocess-exited-with-error × Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> [20 lines of output] Traceback (most recent call last): File "/home/user/conda/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in main() File "/home/user/conda/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main json_out['return_val'] = hook(**hook_input['kwargs']) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/conda/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel return hook(config_settings) ^^^^^^^^^^^^^^^^^^^^^ File "/tmp/pip-build-env-or3gcfq3/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel return self._get_build_requires(config_settings, requirements=['wheel']) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/tmp/pip-build-env-or3gcfq3/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires self.run_setup() File "/tmp/pip-build-env-or3gcfq3/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 487, in run_setup super().run_setup(setup_script=setup_script) File "/tmp/pip-build-env-or3gcfq3/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 311, in run_setup exec(code, locals()) File "", line 9, in ModuleNotFoundError: No module named 'torch' [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error × Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> See above for output. note: This error originates from a subprocess, and is likely not a problem with pip. ```

Question - How solve this problem?

Because as you can see, the imports are in setup.py are almost identical, why can touchvision be installed in the above way, but torchaudio is not

TorchvVision: https://github.com/pytorch/vision/blob/2c127da8b5e2e8f44b50994c6cb931bcca267cfe/setup.py#L9

TorchAudio: https://github.com/pytorch/audio/blob/87aeb554d3e2f7855b7abe5120c282f59648ed7a/setup.py#L9

Versions

PyTorch version: 2.2.0a0 Is debug build: False CUDA used to build PyTorch: 12.2 ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.4 LTS (x86_64) GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 Clang version: Could not collect CMake version: Could not collect Libc version: glibc-2.35

Python version: 3.11.5 (main, Sep 11 2023, 13:54:46) [GCC 11.2.0] (64-bit runtime) Python platform: Linux-6.6.12-linuxkit-x86_64-with-glibc2.35 Is CUDA available: False CUDA runtime version: 12.2.140 CUDA_MODULE_LOADING set to: N/A GPU models and configuration: Could not collect Nvidia driver version: Could not collect cuDNN version: Probably one of the following: /usr/lib/x86_64-linux-gnu/libcudnn.so.8.9.6 /usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.9.6 /usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.9.6 /usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.9.6 /usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.9.6 /usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.9.6 /usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.9.6 HIP runtime version: N/A MIOpen runtime version: N/A Is XNNPACK available: True

CPU: Architecture: x86_64 CPU op-mode(s): 32-bit Byte Order: Little Endian CPU(s): 12 On-line CPU(s) list: 0-11 Vendor ID: 0x00 Model: 0 Thread(s) per core: 1 Core(s) per socket: 12 Socket(s): 1 Stepping: 0x0 BogoMIPS: 48.00 Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 asimddp sha512 asimdfhm dit uscat ilrcpc flagm ssbs sb paca pacg dcpodp flagm2 frint Vulnerability Gather data sampling: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1: Mitigation; __user pointer sanitization Vulnerability Spectre v2: Not affected Vulnerability Srbds: Not affected Vulnerability Tsx async abort: Not affected

Versions of relevant libraries: [pip3] numpy==1.26.0 [pip3] torch==2.2.0a0+gitunknown [pip3] triton==2.1.0 [conda] mkl 2024.0.0 intel_49656 intel [conda] mkl-dpcpp 2024.0.0 intel_49656 intel [conda] mkl-include 2024.0.0 intel_49656 intel [conda] numpy 1.26.0 py311h24aa872_0 anaconda [conda] numpy-base 1.26.0 py311hbfb1bba_0
[conda] onemkl-sycl-blas 2024.0.0 intel_49656 intel [conda] onemkl-sycl-datafitting 2024.0.0 intel_49656 intel [conda] onemkl-sycl-dft 2024.0.0 intel_49656 intel [conda] onemkl-sycl-lapack 2024.0.0 intel_49656 intel [conda] onemkl-sycl-rng 2024.0.0 intel_49656 intel [conda] onemkl-sycl-sparse 2024.0.0 intel_49656 intel [conda] onemkl-sycl-stats 2024.0.0 intel_49656 intel [conda] onemkl-sycl-vm 2024.0.0 intel_49656 intel [conda] torch 2.2.0a0+gitunknown pypi_0 pypi [conda] triton 2.1.0 pypi_0 pypi