search

pytorch / audio

Data manipulation and transformation for audio signal processing, powered by PyTorch
https://pytorch.org/audio
BSD 2-Clause "Simplified" License
2.52k stars 650 forks source link

Not building CUDA 12.6 #3835

Open johnnynunez opened 4 weeks ago

johnnynunez commented 4 weeks ago

🐛 Describe the bug

It's not building with last version of cuda 12.6.1 in jetson agx orin

#!/usr/bin/env bash
set -ex
echo "Building torchaudio ${TORCHAUDIO_VERSION}"

apt-get update
apt-get install -y --no-install-recommends \
        git \
        pkg-config \
        libffi-dev \
        libsndfile1

rm -rf /var/lib/apt/lists/*
apt-get clean

git clone --branch v${TORCHAUDIO_VERSION} --recursive --depth=1 https://github.com/pytorch/audio /opt/torchaudio
cd /opt/torchaudio
git checkout v${TORCHAUDIO_VERSION}

BUILD_SOX=1 python3 setup.py bdist_wheel --verbose --dist-dir /opt

cd ../
rm -rf /opt/torchaudio

pip3 install --no-cache-dir --verbose /opt/torchaudio*.whl
pip3 show torchaudio && python3 -c 'import torchaudio; print(torchaudio.__version__);'

twine upload --verbose /opt/torchaudio*.whl || echo "failed to upload wheel to ${TWINE_REPOSITORY_URL}"
src/include -isystem /usr/local/lib/python3.10/dist-packages/torch/include -isystem /usr/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -isystem /usr/local/cuda/include -Wall -D_GLIBCXX_USE_CXX11_ABI=1 -O3 -DNDEBUG -std=gnu++17 -fPIC -D_GLIBCXX_USE_CXX11_ABI=1 -MD -MT src/libtorio/ffmpeg/CMakeFiles/_torio_ffmpeg4.dir/pybind/pybind.cpp.o -MF src/libtorio/ffmpeg/CMakeFiles/_torio_ffmpeg4.dir/pybind/pybind.cpp.o.d -o src/libtorio/ffmpeg/CMakeFiles/_torio_ffmpeg4.dir/pybind/pybind.cpp.o -c /opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp
In file included from /usr/local/lib/python3.10/dist-packages/torch/include/c10/util/Exception.h:5,
                 from /usr/local/lib/python3.10/dist-packages/torch/include/ATen/BlasBackend.h:3,
                 from /usr/local/lib/python3.10/dist-packages/torch/include/ATen/Context.h:3,
                 from /usr/local/lib/python3.10/dist-packages/torch/include/ATen/ATen.h:7,
                 from /usr/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                 from /opt/torchaudio/src/libtorio/ffmpeg/ffmpeg.h:3,
                 from /opt/torchaudio/src/libtorio/ffmpeg/hw_context.h:3,
                 from /opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp:1:
/opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp: In function ‘int torio::io::{anonymous}::{anonymous}::read_func(void*, uint8_t*, int)’:
/opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp:125:19: warning: comparison of integer expressions of different signedness: ‘long unsigned int’ and ‘int’ [-Wsign-compare]
  125 |         chunk_len <= request,
      |         ~~~~~~~~~~^~~~~~~~~~
In file included from /opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f4-src/include/libavutil/avutil.h:296,
                 from /opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f4-src/include/libavutil/samplefmt.h:24,
                 from /opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f4-src/include/libavcodec/avcodec.h:31,
                 from /opt/torchaudio/src/libtorio/ffmpeg/ffmpeg.h:10,
                 from /opt/torchaudio/src/libtorio/ffmpeg/hw_context.h:3,
                 from /opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp:1:
/opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp: In function ‘int torio::io::{anonymous}::read_bytes(void*, uint8_t*, int)’:
/opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f4-src/include/libavutil/common.h:105:25: warning: comparison of integer expressions of different signedness: ‘std::basic_string_view<char>::size_type’ {aka ‘long unsigned int’} and ‘int’ [-Wsign-compare]
  105 | #define FFMIN(a,b) ((a) > (b) ? (b) : (a))
      |                     ~~~~^~~~~
/opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp:202:19: note: in expansion of macro ‘FFMIN’
  202 |   auto num_read = FFMIN(wrapper->src.size() - wrapper->index, buf_size);
      |                   ^~~~~
[82/92] /usr/bin/c++ -DTORIO_FFMPEG_EXT_NAME=_torio_ffmpeg6 -DUSE_C10D_GLOO -DUSE_C10D_MPI -DUSE_C10D_NCCL -DUSE_CUDA -DUSE_DISTRIBUTED -DUSE_RPC -DUSE_TENSORPIPE -D_torio_ffmpeg6_EXPORTS -I/opt/torchaudio/src -I/usr/include/python3.10 -I/opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f6-src/include -isystem /usr/local/lib/python3.10/dist-packages/torch/include -isystem /usr/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -isystem /usr/local/cuda/include -Wall -D_GLIBCXX_USE_CXX11_ABI=1 -O3 -DNDEBUG -std=gnu++17 -fPIC -D_GLIBCXX_USE_CXX11_ABI=1 -MD -MT src/libtorio/ffmpeg/CMakeFiles/_torio_ffmpeg6.dir/pybind/pybind.cpp.o -MF src/libtorio/ffmpeg/CMakeFiles/_torio_ffmpeg6.dir/pybind/pybind.cpp.o.d -o src/libtorio/ffmpeg/CMakeFiles/_torio_ffmpeg6.dir/pybind/pybind.cpp.o -c /opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp
In file included from /usr/local/lib/python3.10/dist-packages/torch/include/c10/util/Exception.h:5,
                 from /usr/local/lib/python3.10/dist-packages/torch/include/ATen/BlasBackend.h:3,
                 from /usr/local/lib/python3.10/dist-packages/torch/include/ATen/Context.h:3,
                 from /usr/local/lib/python3.10/dist-packages/torch/include/ATen/ATen.h:7,
                 from /usr/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                 from /opt/torchaudio/src/libtorio/ffmpeg/ffmpeg.h:3,
                 from /opt/torchaudio/src/libtorio/ffmpeg/hw_context.h:3,
                 from /opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp:1:
/opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp: In function ‘int torio::io::{anonymous}::{anonymous}::read_func(void*, uint8_t*, int)’:
/opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp:125:19: warning: comparison of integer expressions of different signedness: ‘long unsigned int’ and ‘int’ [-Wsign-compare]
  125 |         chunk_len <= request,
      |         ~~~~~~~~~~^~~~~~~~~~
In file included from /opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f6-src/include/libavutil/common.h:43,
                 from /opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f6-src/include/libavutil/avutil.h:296,
                 from /opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f6-src/include/libavcodec/avcodec.h:32,
                 from /opt/torchaudio/src/libtorio/ffmpeg/ffmpeg.h:10,
                 from /opt/torchaudio/src/libtorio/ffmpeg/hw_context.h:3,
                 from /opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp:1:
/opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp: In function ‘int torio::io::{anonymous}::read_bytes(void*, uint8_t*, int)’:
/opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f6-src/include/libavutil/macros.h:49:25: warning: comparison of integer expressions of different signedness: ‘std::basic_string_view<char>::size_type’ {aka ‘long unsigned int’} and ‘int’ [-Wsign-compare]
   49 | #define FFMIN(a,b) ((a) > (b) ? (b) : (a))
      |                     ~~~~^~~~~
/opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp:202:19: note: in expansion of macro ‘FFMIN’
  202 |   auto num_read = FFMIN(wrapper->src.size() - wrapper->index, buf_size);
      |                   ^~~~~
[83/92] /usr/bin/c++ -DTORIO_FFMPEG_EXT_NAME=_torio_ffmpeg5 -DUSE_C10D_GLOO -DUSE_C10D_MPI -DUSE_C10D_NCCL -DUSE_CUDA -DUSE_DISTRIBUTED -DUSE_RPC -DUSE_TENSORPIPE -D_torio_ffmpeg5_EXPORTS -I/opt/torchaudio/src -I/usr/include/python3.10 -I/opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f5-src/include -isystem /usr/local/lib/python3.10/dist-packages/torch/include -isystem /usr/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -isystem /usr/local/cuda/include -Wall -D_GLIBCXX_USE_CXX11_ABI=1 -O3 -DNDEBUG -std=gnu++17 -fPIC -D_GLIBCXX_USE_CXX11_ABI=1 -MD -MT src/libtorio/ffmpeg/CMakeFiles/_torio_ffmpeg5.dir/pybind/pybind.cpp.o -MF src/libtorio/ffmpeg/CMakeFiles/_torio_ffmpeg5.dir/pybind/pybind.cpp.o.d -o src/libtorio/ffmpeg/CMakeFiles/_torio_ffmpeg5.dir/pybind/pybind.cpp.o -c /opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp
In file included from /usr/local/lib/python3.10/dist-packages/torch/include/c10/util/Exception.h:5,
                 from /usr/local/lib/python3.10/dist-packages/torch/include/ATen/BlasBackend.h:3,
                 from /usr/local/lib/python3.10/dist-packages/torch/include/ATen/Context.h:3,
                 from /usr/local/lib/python3.10/dist-packages/torch/include/ATen/ATen.h:7,
                 from /usr/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                 from /opt/torchaudio/src/libtorio/ffmpeg/ffmpeg.h:3,
                 from /opt/torchaudio/src/libtorio/ffmpeg/hw_context.h:3,
                 from /opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp:1:
/opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp: In function ‘int torio::io::{anonymous}::{anonymous}::read_func(void*, uint8_t*, int)’:
/opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp:125:19: warning: comparison of integer expressions of different signedness: ‘long unsigned int’ and ‘int’ [-Wsign-compare]
  125 |         chunk_len <= request,
      |         ~~~~~~~~~~^~~~~~~~~~
In file included from /opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f5-src/include/libavutil/common.h:43,
                 from /opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f5-src/include/libavutil/avutil.h:296,
                 from /opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f5-src/include/libavutil/samplefmt.h:24,
                 from /opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f5-src/include/libavcodec/avcodec.h:30,
                 from /opt/torchaudio/src/libtorio/ffmpeg/ffmpeg.h:10,
                 from /opt/torchaudio/src/libtorio/ffmpeg/hw_context.h:3,
                 from /opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp:1:
/opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp: In function ‘int torio::io::{anonymous}::read_bytes(void*, uint8_t*, int)’:
/opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f5-src/include/libavutil/macros.h:49:25: warning: comparison of integer expressions of different signedness: ‘std::basic_string_view<char>::size_type’ {aka ‘long unsigned int’} and ‘int’ [-Wsign-compare]
   49 | #define FFMIN(a,b) ((a) > (b) ? (b) : (a))
      |                     ~~~~^~~~~
/opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp:202:19: note: in expansion of macro ‘FFMIN’
  202 |   auto num_read = FFMIN(wrapper->src.size() - wrapper->index, buf_size);
      |                   ^~~~~
ninja: build stopped: subcommand failed.

Traceback (most recent call last):
  File "/opt/torchaudio/setup.py", line 143, in <module>
    _main()
  File "/opt/torchaudio/setup.py", line 99, in _main
    setup(
  File "/usr/local/lib/python3.10/dist-packages/setuptools/__init__.py", line 117, in setup
    return distutils.core.setup(**attrs)
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/core.py", line 183, in setup
    return run_commands(dist)
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/core.py", line 199, in run_commands
    dist.run_commands()
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/dist.py", line 954, in run_commands
    self.run_command(cmd)
  File "/usr/local/lib/python3.10/dist-packages/setuptools/dist.py", line 950, in run_command
    super().run_command(command)
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/dist.py", line 973, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.10/dist-packages/setuptools/command/bdist_wheel.py", line 398, in run
    self.run_command("build")
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/cmd.py", line 316, in run_command
    self.distribution.run_command(command)
  File "/usr/local/lib/python3.10/dist-packages/setuptools/dist.py", line 950, in run_command
    super().run_command(command)
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/dist.py", line 973, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/cmd.py", line 316, in run_command
    self.distribution.run_command(command)
  File "/usr/local/lib/python3.10/dist-packages/setuptools/dist.py", line 950, in run_command
    super().run_command(command)
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/dist.py", line 973, in run_command
    cmd_obj.run()
  File "/opt/torchaudio/tools/setup_helpers/extension.py", line 99, in run
    super().run()
  File "/usr/local/lib/python3.10/dist-packages/setuptools/command/build_ext.py", line 98, in run
    _build_ext.run(self)
  File "/usr/local/lib/python3.10/dist-packages/Cython/Distutils/old_build_ext.py", line 186, in run
    _build_ext.build_ext.run(self)
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/command/build_ext.py", line 359, in run
    self.build_extensions()
  File "/usr/local/lib/python3.10/dist-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
    _build_ext.build_ext.build_extensions(self)
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/command/build_ext.py", line 476, in build_extensions
    self._build_extensions_serial()
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/command/build_ext.py", line 502, in _build_extensions_serial
    self.build_extension(ext)
  File "/opt/torchaudio/tools/setup_helpers/extension.py", line 183, in build_extension
    subprocess.check_call(["cmake", "--build", "."] + build_args, cwd=self.build_temp)
  File "/usr/lib/python3.10/subprocess.py", line 369, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--target', 'install']' returned non-zero exit status 1.
The command '/bin/sh -c /tmp/torchaudio/install.sh || /tmp/torchaudio/build.sh' returned a non-zero code: 1
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/johnny/Projects/jetson-containers/jetson_containers/build.py", line 112, in <module>
    build_container(args.name, args.packages, args.base, args.build_flags, args.build_args, args.simulate, args.skip_tests, args.test_only, args.push, args.no_github_api)
  File "/home/johnny/Projects/jetson-containers/jetson_containers/container.py", line 147, in build_container
    status = subprocess.run(cmd.replace(_NEWLINE_, ' '), executable='/bin/bash', shell=True, check=True)  
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/johnny/miniconda3/lib/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'sudo DOCKER_BUILDKIT=0 docker build --network=host --tag torchaudio:r36.3.0-cu126-cp310-torchaudio --file /home/johnny/Projects/jetson-containers/packages/pytorch/torchaudio/Dockerfile --build-arg BASE_IMAGE=torchaudio:r36.3.0-cu126-cp310-pytorch_2.4 --build-arg TORCHAUDIO_VERSION="2.4.0" /home/johnny/Projects/jetson-containers/packages/pytorch/torchaudio 2>&1 | tee /home/johnny/Projects/jetson-containers/logs/20240925_120647/build/torchaudio_r36.3.0-cu126-cp310-torchaudio.txt; exit ${PIPESTATUS[0]}' returned non-zero exit status 1.

Versions

Pytorch 2.4.0

williamstar commented 3 days ago

Not building cuda 12.5 too