search

pytorch / audio

Data manipulation and transformation for audio signal processing, powered by PyTorch
https://pytorch.org/audio
BSD 2-Clause "Simplified" License
2.54k stars 652 forks source link

Building torchaudio 0.7 on `ppc64le` #1796

Closed j4sonzhao closed 3 years ago

j4sonzhao commented 3 years ago

Hi there, I am trying to install torchaudio from source, and I run into the following error:

CMake Error at [...]/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:58 (message):
  Your installed Caffe2 version uses protobuf but the protobuf library cannot
  be found.  Did you accidentally remove it, or have you set the right
  CMAKE_PREFIX_PATH? If you do not have protobuf, you will need to install
  protobuf and set the library path accordingly.

I think I have protobuf installed because when I run pip install protobuf I get

Requirement already satisfied: protobuf in [...]/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages (3.9.2)
Requirement already satisfied: six>=1.9 in [...]/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages (from protobuf) (1.15.0)
Requirement already satisfied: setuptools in [...]/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages (from protobuf) (58.0.4)

I've tried modifying the CMAKE_PREFIX_PATH but I don't really know what I'm doing there, and I'm not sure if that's the right way to go.

Help would be appreciated! I need to install from source since there's no other way I can install on the machine I'm using right now.

nateanl commented 3 years ago

Thanks for the report. Could you post your conda and PyTorch information here? You can do it by copy and paste the output from environment collection script

j4sonzhao commented 3 years ago

@nateanl

I'm running torch 1.7.1 and python 3.7. I'll post the info from running the script later tonight or tomorrow; I'm just not able to do so now.

j4sonzhao commented 3 years ago

Another thing to note is that I am running on a ppc64le (power 9) machine

nateanl commented 3 years ago

Maybe upgrading torch to 1.9 can solve the problem, but I'm not sure. I will try to reproduce it when you post the environment info. Also could you paste the command you use for building torchaudio from source? Thanks.

mthrok commented 3 years ago

From the error message, I think you are building torchaudio from the main branch. This requires the latest PyTorch (nightly or source build).

With PyTorch 1.7.1, you need to use the v0.7.2 tag. https://github.com/pytorch/audio/tree/v0.7.2, ~which does not use cmake for building~ (EDIT it does use cmake for third party, but not for torchaudio extension).

I am not sure about ppc64le environment, but if you successfully installed PyTorch 1.7.1, there is a good chance that torchaudio v0.7.2 can be installed.

j4sonzhao commented 3 years ago

Thanks, will give this a try today!

j4sonzhao commented 3 years ago

@mthrok @nateanl

Thanks! Building from the above tag instead seemed to fix the error. However, it looks like I am now getting a new error when building from source using python setup.py install, and it's very difficult for me to parse the error. The relevant part of the error is pasted below:

FAILED: /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/register.o 
c++ -MMD -MF /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/register.o.d -pthread -B /nobackup/users/jzhao7/anaconda3/envs/p
ytorch1.7real/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jzhao7/audio -I/nobackup/users/jzh
ao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/
torch/include/torch/csrc/api/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/TH -I/nobackup/users/jzha
o7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/THC -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/include/python3.7m -c -c 
/home/jzhao7/audio/torchaudio/csrc/register.cpp -o /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/register.o -O3 -DTORCH_API
_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_torchaudio
 -D_GLIBCXX_USE_CXX11_ABI=1 -std=gnu++14
    env=env)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

and here is the full error message:

No CUDA runtime is found, using CUDA_HOME='/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real'
-- Building version 0.7.0a0+a853dff
running install
running bdist_egg
running egg_info
writing torchaudio.egg-info/PKG-INFO
writing dependency_links to torchaudio.egg-info/dependency_links.txt
writing requirements to torchaudio.egg-info/requires.txt
writing top-level names to torchaudio.egg-info/top_level.txt
reading manifest file 'torchaudio.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'torchaudio.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-ppc64le/egg
running install_lib
running build_py
copying torchaudio/version.py -> build/lib.linux-ppc64le-3.7/torchaudio
running build_ext
building 'torchaudio._torchaudio' extension
/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/utils/cpp_extension.py:269: UserWarning: 

                               !! WARNING !!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Your compiler (c++) is not compatible with the compiler Pytorch was
built with for this platform, which is g++ on linux. Please
use g++ to to compile your extension. Alternatively, you may
compile PyTorch from source using c++, and then you can also use
c++ to compile your extension.

See https://github.com/pytorch/pytorch/blob/master/CONTRIBUTING.md for help
with compiling PyTorch from source.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

                              !! WARNING !!

  platform=sys.platform))
Emitting ninja build file /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/6] c++ -MMD -MF /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox_io.o.d -pthread -B /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jzhao7/audio -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/TH -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/THC -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/include/python3.7m -c -c /home/jzhao7/audio/torchaudio/csrc/sox_io.cpp -o /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox_io.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_torchaudio -D_GLIBCXX_USE_CXX11_ABI=1 -std=gnu++14
FAILED: /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox_io.o 
c++ -MMD -MF /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox_io.o.d -pthread -B /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jzhao7/audio -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/TH -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/THC -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/include/python3.7m -c -c /home/jzhao7/audio/torchaudio/csrc/sox_io.cpp -o /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox_io.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_torchaudio -D_GLIBCXX_USE_CXX11_ABI=1 -std=gnu++14
-D_GLIBCXX_USE_CXX11_ABI=1 -std=gnu++14
1_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_torchaudio -D_GLIBCXX_USE_CXX11_ABI=1 -std=gnu++14
FAILED: /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox.o 
c++ -MMD -MF /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox.o.d -pthread -B /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jzhao7/audio -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/TH -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/THC -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/include/python3.7m -c -c /home/jzhao7/audio/torchaudio/csrc/sox.cpp -o /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_torchaudio -D_GLIBCXX_USE_CXX11_ABI=1 -std=gnu++14
[5/6] c++ -MMD -MF /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox_utils.o.d -pthread -B /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jzhao7/audio -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/TH -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/THC -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/include/python3.7m -c -c /home/jzhao7/audio/torchaudio/csrc/sox_utils.cpp -o /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox_utils.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_torchaudio -D_GLIBCXX_USE_CXX11_ABI=1 -std=gnu++14
FAILED: /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox_utils.o 
c++ -MMD -MF /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox_utils.o.d -pthread -B /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jzhao7/audio -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/TH -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/THC -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/include/python3.7m -c -c /home/jzhao7/audio/torchaudio/csrc/sox_utils.cpp -o /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox_utils.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_torchaudio -D_GLIBCXX_USE_CXX11_ABI=1 -std=gnu++14
[6/6] c++ -MMD -MF /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/register.o.d -pthread -B /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jzhao7/audio -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/TH -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/THC -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/include/python3.7m -c -c /home/jzhao7/audio/torchaudio/csrc/register.cpp -o /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/register.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_torchaudio -D_GLIBCXX_USE_CXX11_ABI=1 -std=gnu++14
FAILED: /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/register.o 
c++ -MMD -MF /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/register.o.d -pthread -B /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jzhao7/audio -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/TH -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/THC -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/include/python3.7m -c -c /home/jzhao7/audio/torchaudio/csrc/register.cpp -o /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/register.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_torchaudio -D_GLIBCXX_USE_CXX11_ABI=1 -std=gnu++14
    env=env)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "setup.py", line 89, in <module>
    zip_safe=False,
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/install.py", line 67, in run
    self.do_egg_install()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/install.py", line 109, in do_egg_install
    self.run_command('bdist_egg')
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 164, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 150, in call_command
    self.run_command(cmdname)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/install_lib.py", line 11, in run
    self.build()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/install_lib.py", line 107, in build
running build_ext
    self.run_command('build_ext')
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 79, in run
    _build_ext.run(self)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 677, in build_extensions
    build_ext.build_extensions(self)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
    self._build_extensions_serial()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
    self.build_extension(ext)
  File "/home/jzhao7/audio/build_tools/setup_helpers/extension.py", line 140, in build_extension
    super().build_extension(ext)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 202, in build_extension
    _build_ext.build_extension(self, ext)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/build_ext.py", line 534, in build_extension
    depends=ext.depends)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 507, in unix_wrap_ninja_compile
    with_cuda=with_cuda)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1262, in _write_ninja_file_and_com
pile_objects
    error_prefix='Error compiling objects for extension')
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1562, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error compiling objects for extension

There is a warning saying something about c++ compiler compatibility, could this be relevant here? There is also a message that says that the compute node I'm currently on doesn't see a CUDA path (maybe this is a function of the compute node that I'm currently on; torch.cuda.is_available is currently showing false), so not sure if this could be relevant as well?

Also, if I run g++ --version I get g++ 8.3.1, if that's relevant at all.

Thanks so much!

mthrok commented 3 years ago

FAILED: /home/<snip>/torchaudio/csrc/register.o

The error indicates that it failed to compile torchaudio/csrc/register.cpp, however it does not show any message about why it failed. In my experience, this is often caused by OOM, even though other causes are still highly possible. Can you try setting MAX_JOBS=1?

Also since the underlying compile command is shown, can you try running the compile command directly? it's c++ -MMD -MF ... after [6/6]

There is a warning saying something about c++ compiler compatibility, could this be relevant here?

It is possible, but I expect something strange to happen at link time or run time.

doesn't see a CUDA path

This should not be a problem for building torchaduio, because torchaudio 0.7 does not have any custom CUDA kernel.

j4sonzhao commented 3 years ago

@mthrok

I get a compilation error when I try to run the compile command directly:

(pytorch1.7real) [jzhao7@node0033 audio]$ c++ -MMD -MF /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox_io.o.d -pthread -B
 /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I
/home/jzhao7/audio -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include -I/nobackup/users/jzhao7/anaconda3/envs/pyt
orch1.7real/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages
/torch/include/TH -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/THC -I/nobackup/users/jzhao7/anaconda3/envs/
pytorch1.7real/include/python3.7m -c -c /home/jzhao7/audio/torchaudio/csrc/sox_io.cpp -o /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torc
haudio/csrc/sox_io.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"
' -DTORCH_EXTENSION_NAME=_torchaudio -D_GLIBCXX_USE_CXX11_ABI=1 -std=gnu++14
cc1plus: warning: command line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++
/home/jzhao7/audio/torchaudio/csrc/sox_io.cpp:1:10: fatal error: sox.h: No such file or directory
 #include <sox.h>
          ^~~~~~~
compilation terminated.

This is the command that comes after [1/6], but I get the same errors when I run the commands that come after [5/6] and [6/6].

Could you please specify what you mean by setting MAX_JOBS=1? I simply just ran export MAX_JOBS=1 before running python setup.py install and got the same result.

Thanks so much!

mthrok commented 3 years ago

Cool, now we are making progress and know what is going on. The error indicates that the system does not have the library Torchaudio depends on. We have two options, building libsox as a part of Torchaudio, or use externally managed library and link dynamically.

If your OS has a package manager and if libsox is available, then you can use that. You need to do some equivalent of APT command like sudo apt-get install sox libsox-fmt-all. This will install libsox and all the codec plugins and Torchaudio's build process will find it during the build process.

Another option, to build sox as a part of Torchaudio, you can set the environment variable BUILD_SOX=1 (export BUILD_SOX=1), then this will download the source code of libsox and other libraries and build them. The problem is that we do not know if this works for ppc64le. Lucky if it works out-of-box.

Could you please specify what you mean by setting MAX_JOBS=1? I simply just ran export MAX_JOBS=1 before running python setup.py install and got the same result.

Yes, I meant the environment variable as you did, and since the compilation error was not OOM, indeed this did not help.

j4sonzhao commented 3 years ago

Just ran the setup command with BUILD_SOX=1, and around half of the modules managed to install, except halfway through it threw this error

[ 54%] No update step for 'libmp3lame'
[ 56%] No patch step for 'libmp3lame'
[ 57%] Performing configure step for 'libmp3lame'
configure: WARNING: unrecognized options: --disable-examples, --disable-doc
/home/jzhao7/audio/third_party/src/libmp3lame/config.guess: unable to guess system type

This script, last modified 2003-07-02, has failed to recognize
the operating system you are using. It is advised that you
download the most up to date version of the config scripts from

    ftp://ftp.gnu.org/pub/gnu/config/

If the version you run (/home/jzhao7/audio/third_party/src/libmp3lame/config.guess) is already up to date, please
send the following data and any information you think might be
pertinent to <config-patches@gnu.org> in order to provide the needed
information to handle your system.

config.guess timestamp = 2003-07-02

uname -m = ppc64le
uname -r = 4.18.0-240.el8.ppc64le
uname -s = Linux
uname -v = #1 SMP Wed Sep 23 05:08:15 EDT 2020

/usr/bin/uname -p = ppc64le
/bin/uname -X     = 

hostinfo               = 
/bin/universe          = 
/usr/bin/arch -k       = 
/bin/arch              = ppc64le
/usr/bin/oslevel       = 
/usr/convex/getsysinfo = 

UNAME_MACHINE = ppc64le
UNAME_RELEASE = 4.18.0-240.el8.ppc64le
UNAME_SYSTEM  = Linux
UNAME_VERSION = #1 SMP Wed Sep 23 05:08:15 EDT 2020
configure: error: cannot guess build type; you must specify one
gmake[2]: *** [CMakeFiles/libmp3lame.dir/build.make:92: ../src/libmp3lame-stamp/libmp3lame-configure] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:154: CMakeFiles/libmp3lame.dir/all] Error 2
gmake: *** [Makefile:91: all] Error 2
Traceback (most recent call last):
  File "setup.py", line 89, in <module>
    zip_safe=False,
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/install.py", line 67, in run
    self.do_egg_install()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/install.py", line 109, in do_egg_install
    self.run_command('bdist_egg')
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 164, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 150, in call_command
    self.run_command(cmdname)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
uname -m = ppc64le
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/install_lib.py", line 11, in run
    self.build()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/install_lib.py", line 107, in build
    self.run_command('build_ext')
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 79, in run
    _build_ext.run(self)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 677, in build_extensions
    build_ext.build_extensions(self)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
    self._build_extensions_serial()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
    self.build_extension(ext)
  File "/home/jzhao7/audio/build_tools/setup_helpers/extension.py", line 139, in build_extension
    _build_third_party()
  File "/home/jzhao7/audio/build_tools/setup_helpers/extension.py", line 113, in _build_third_party
    check=True,
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['cmake', '--build', '.']' returned non-zero exit status 2.

I assume that this means it doesn't work for ppc64le, and I have to do the installation of libsox through the command line? I will get back to you tomorrow once I do this - I currently don't have sudo permissions on this machine, so I will ask a sysadmin about this.

mthrok commented 3 years ago

@j4sonzhao

Yes, this is what I suspected to happen. The source code of libsox is years old, so they do not detect recent CPU architectures well. ~There is still workaround for this, but I just realized that, since you are using Anaconda, there might be a sox package you can install.~ (UPDATE: this does not seems to work https://docs.anaconda.com/anaconda/packages/py3.6_linux-ppc64le/) But still can you try conda install sox just in case?

The same error also happened when we tried building on macOS with M1 chip. One way to resolve the error is to patch the lame and libmad. This is practically same as https://github.com/pytorch/audio/pull/1613.

  1. Download config.guess and config.sub and put it in third_parth/patch directory.
  2. In third_parth/CMakeLists.txt, add/update the PATCH_COMMAND from the latest CMakeLists.txt. Specifically, the following two lines
  3. Try build with BUILD_SOX=1.
j4sonzhao commented 3 years ago

@mthrok

I tried what you said above, and it seems that I am getting some sort of error where it seems that the path is misspecified? (Also, the last time I ran with build_sox=1 the installation quit halfway through after some packages had already finished; not sure if that matters) Here is the full error:

No CUDA runtime is found, using CUDA_HOME='/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real'
-- Building version 0.7.0a0+a853dff
running install
running bdist_egg
running egg_info
writing torchaudio.egg-info/PKG-INFO
writing dependency_links to torchaudio.egg-info/dependency_links.txt
writing requirements to torchaudio.egg-info/requires.txt
writing top-level names to torchaudio.egg-info/top_level.txt
reading manifest file 'torchaudio.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'torchaudio.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-ppc64le/egg
running install_lib
running build_py
copying torchaudio/version.py -> build/lib.linux-ppc64le-3.7/torchaudio
running build_ext
-- Configuring done
-- Generating done
-- Build files have been written to: /home/jzhao7/audio/third_party/build
[ 12%] Built target libogg
[ 25%] Built target libflac
[ 37%] Built target libopus
[ 50%] Built target opusfile
[ 51%] Performing patch step for 'libmp3lame'
cp: cannot create regular file '/home/jzhao7/audio/third_party/build/src/lame/config.guess': No such file or directory
gmake[2]: *** [CMakeFiles/libmp3lame.dir/build.make:121: ../src/libmp3lame-stamp/libmp3lame-patch] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:154: CMakeFiles/libmp3lame.dir/all] Error 2
gmake: *** [Makefile:91: all] Error 2
Traceback (most recent call last):
  File "setup.py", line 89, in <module>
    zip_safe=False,
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/install.py", line 67, in run
    self.do_egg_install()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/install.py", line 109, in do_egg_install
    self.run_command('bdist_egg')
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 164, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 150, in call_command
    self.run_command(cmdname)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/install_lib.py", line 11, in run
    self.build()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/install_lib.py", line 107, in build
    self.run_command('build_ext')
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 79, in run
    _build_ext.run(self)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 677, in build_extensions
    build_ext.build_extensions(self)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
    self._build_extensions_serial()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
    self.build_extension(ext)
  File "/home/jzhao7/audio/build_tools/setup_helpers/extension.py", line 139, in build_extension
    _build_third_party()
  File "/home/jzhao7/audio/build_tools/setup_helpers/extension.py", line 113, in _build_third_party
    check=True,
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['cmake', '--build', '.']' returned non-zero exit status 2.

Here is what my third_party/patch directory looks like: image and I verified that the config.guess and config.sub matches up with what you posted.

And here is what my third_party/patch/CMakeLists.txt looks like. It should be the same as before, except I changed the PATCH command of the first 2 packages, libmad and libmp3lame.

cmake_minimum_required(VERSION 3.1)

project(torchaudio_third_parties)
include(ExternalProject)

set(INSTALL_DIR ${CMAKE_CURRENT_SOURCE_DIR}/install)
set(ARCHIVE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/archives)
set(COMMON_ARGS --quiet --disable-shared --enable-static --prefix=${INSTALL_DIR} --with-pic --disable-dependency-tracking --disable-debug --disable-examples --disable-doc)

ExternalProject_Add(libmad
  PREFIX ${CMAKE_CURRENT_SOURCE_DIR}
  DOWNLOAD_DIR ${ARCHIVE_DIR}
  URL https://downloads.sourceforge.net/project/mad/libmad/0.15.1b/libmad-0.15.1b.tar.gz
  URL_HASH SHA256=bbfac3ed6bfbc2823d3775ebb931087371e142bb0e9bb1bee51a76a6e0078690
  PATCH_COMMAND patch < ${CMAKE_CURRENT_SOURCE_DIR}/patch/libmad.patch && cp ${CMAKE_CURRENT_SOURCE_DIR}/patch/config.guess ${CMAKE_CURRENT_BINARY_DIR}/src/mad/config.guess && cp ${CMAKE_CURRENT_SOURCE_DIR}/patch/config.sub ${CMAKE_CURRENT_BINARY_DIR}/src/mad/config.sub
  CONFIGURE_COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/src/libmad/configure ${COMMON_ARGS}
)

ExternalProject_Add(libmp3lame
  PREFIX ${CMAKE_CURRENT_SOURCE_DIR}
  DOWNLOAD_DIR ${ARCHIVE_DIR}
  URL https://downloads.sourceforge.net/project/lame/lame/3.99/lame-3.99.5.tar.gz
  URL_HASH SHA256=24346b4158e4af3bd9f2e194bb23eb473c75fb7377011523353196b19b9a23ff
  PATCH_COMMAND cp ${CMAKE_CURRENT_SOURCE_DIR}/patch/config.guess ${CMAKE_CURRENT_BINARY_DIR}/src/lame/config.guess && cp ${CMAKE_CURRENT_SOURCE_DIR}/patch/config.sub ${CMAKE_CURRENT_BINARY_DIR}/src/lame/config.sub
  CONFIGURE_COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/src/libmp3lame/configure ${COMMON_ARGS} --enable-nasm
)

ExternalProject_Add(libogg
  PREFIX ${CMAKE_CURRENT_SOURCE_DIR}
  DOWNLOAD_DIR ${ARCHIVE_DIR}
  URL https://ftp.osuosl.org/pub/xiph/releases/ogg/libogg-1.3.3.tar.gz
  URL_HASH SHA256=c2e8a485110b97550f453226ec644ebac6cb29d1caef2902c007edab4308d985
  CONFIGURE_COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/src/libogg/configure ${COMMON_ARGS}
)

ExternalProject_Add(libflac
  PREFIX ${CMAKE_CURRENT_SOURCE_DIR}
  DEPENDS libogg
  DOWNLOAD_DIR ${ARCHIVE_DIR}
  URL https://ftp.osuosl.org/pub/xiph/releases/flac/flac-1.3.2.tar.xz
  URL_HASH SHA256=91cfc3ed61dc40f47f050a109b08610667d73477af6ef36dcad31c31a4a8d53f
  CONFIGURE_COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/build_codec_helper.sh ${CMAKE_CURRENT_SOURCE_DIR}/src/libflac/configure ${COMMON_ARGS} --with-ogg
)

ExternalProject_Add(libvorbis
  PREFIX ${CMAKE_CURRENT_SOURCE_DIR}
  DEPENDS libogg
  DOWNLOAD_DIR ${ARCHIVE_DIR}
  URL https://ftp.osuosl.org/pub/xiph/releases/vorbis/libvorbis-1.3.6.tar.gz
  URL_HASH SHA256=6ed40e0241089a42c48604dc00e362beee00036af2d8b3f46338031c9e0351cb
  CONFIGURE_COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/build_codec_helper.sh ${CMAKE_CURRENT_SOURCE_DIR}/src/libvorbis/configure ${COMMON_ARGS} --with-ogg
)

ExternalProject_Add(libopus
  PREFIX ${CMAKE_CURRENT_SOURCE_DIR}
  DEPENDS libogg
  DOWNLOAD_DIR ${ARCHIVE_DIR}
  URL https://ftp.osuosl.org/pub/xiph/releases/opus/opus-1.3.1.tar.gz
  URL_HASH SHA256=65b58e1e25b2a114157014736a3d9dfeaad8d41be1c8179866f144a2fb44ff9d
  CONFIGURE_COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/build_codec_helper.sh ${CMAKE_CURRENT_SOURCE_DIR}/src/libopus/configure ${COMMON_ARGS} --with-ogg
)

ExternalProject_Add(opusfile
  PREFIX ${CMAKE_CURRENT_SOURCE_DIR}
  DEPENDS libopus
  DOWNLOAD_DIR ${ARCHIVE_DIR}
  STAMP_DIR ${CMAKE_CURRENT_SOURCE_DIR}/src/opusfile-stamp
  SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/src/opusfile
  URL https://ftp.osuosl.org/pub/xiph/releases/opus/opusfile-0.12.tar.gz
  URL_HASH SHA256=118d8601c12dd6a44f52423e68ca9083cc9f2bfe72da7a8c1acb22a80ae3550b
  CONFIGURE_COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/build_codec_helper.sh ${CMAKE_CURRENT_SOURCE_DIR}/src/opusfile/configure ${COMMON_ARGS} --disable-http
)

ExternalProject_Add(libsox
  PREFIX ${CMAKE_CURRENT_SOURCE_DIR}
  DEPENDS libogg libflac libvorbis opusfile libmp3lame libmad
  DOWNLOAD_DIR ${ARCHIVE_DIR}
  URL https://downloads.sourceforge.net/project/sox/sox/14.4.2/sox-14.4.2.tar.bz2
  URL_HASH SHA256=81a6956d4330e75b5827316e44ae381e6f1e8928003c6aa45896da9041ea149c
  # OpenMP is by default compiled against GNU OpenMP, which conflicts with the version of OpenMP that PyTorch uses.
  # See https://github.com/pytorch/audio/pull/1026
  CONFIGURE_COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/build_codec_helper.sh ${CMAKE_CURRENT_SOURCE_DIR}/src/libsox/configure ${COMMON_ARGS} --with-lame --with-flac --with-mad --with-oggvorbis --without-alsa --without-coreaudio --without-png --without-oss --without-sndfile --with-opus --disable-openmp
)

Btw, conda install sox doesn't seem to work.

j4sonzhao commented 3 years ago

@mthrok bumping this :)

mthrok commented 3 years ago

Hi @j4sonzhao

Sorry for the delay. In the PATCH command you added, Can you replace CMAKE_CURRENT_BINARY_DIR with CMAKE_CURRENT_SOURCE_DIR?

 PATCH_COMMAND patch < ${CMAKE_CURRENT_SOURCE_DIR}/patch/libmad.patch && cp ${CMAKE_CURRENT_SOURCE_DIR}/patch/config.guess ${CMAKE_CURRENT_BINARY_DIR}/src/mad/config.guess && cp ${CMAKE_CURRENT_SOURCE_DIR}/patch/config.sub ${CMAKE_CURRENT_BINARY_DIR}/src/mad/config.sub 
 PATCH_COMMAND cp ${CMAKE_CURRENT_SOURCE_DIR}/patch/config.guess ${CMAKE_CURRENT_SOURCE_DIR}/src/lame/config.guess && cp ${CMAKE_CURRENT_SOURCE_DIR}/patch/config.sub ${CMAKE_CURRENT_SOURCE_DIR}/src/lame/config.sub
j4sonzhao commented 3 years ago

@mthrok

I tried that, and cam across a very similar error (I think the path is just slightly different this time, cp: cannot create regular file '/home/jzhao7/audio/third_party/src/lame/config.guess': No such file or directory)

No CUDA runtime is found, using CUDA_HOME='/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real'
-- Building version 0.7.0a0+a853dff
running install
running bdist_egg
running egg_info
writing torchaudio.egg-info/PKG-INFO
writing dependency_links to torchaudio.egg-info/dependency_links.txt
writing requirements to torchaudio.egg-info/requires.txt
writing top-level names to torchaudio.egg-info/top_level.txt
reading manifest file 'torchaudio.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'torchaudio.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-ppc64le/egg
running install_lib
running build_py
copying torchaudio/version.py -> build/lib.linux-ppc64le-3.7/torchaudio
running build_ext
-- Configuring done
-- Generating done
-- Build files have been written to: /home/jzhao7/audio/third_party/build
[ 12%] Built target libogg
[ 25%] Built target libflac
[ 37%] Built target libopus
[ 50%] Built target opusfile
[ 51%] Performing patch step for 'libmp3lame'
cp: cannot create regular file '/home/jzhao7/audio/third_party/src/lame/config.guess': No such file or directory
gmake[2]: *** [CMakeFiles/libmp3lame.dir/build.make:121: ../src/libmp3lame-stamp/libmp3lame-patch] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:154: CMakeFiles/libmp3lame.dir/all] Error 2
gmake: *** [Makefile:91: all] Error 2
Traceback (most recent call last):
  File "setup.py", line 89, in <module>
    zip_safe=False,
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 985, in run_command
    dist.run_commands()
    cmd_obj.run()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/install.py", line 67, in run
    self.do_egg_install()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/install.py", line 109, in do_egg_install
    self.run_command('bdist_egg')
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 164, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 150, in call_command
    self.run_command(cmdname)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/install_lib.py", line 11, in run
    self.build()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/install_lib.py", line 107, in build
    self.run_command('build_ext')
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 79, in run
    _build_ext.run(self)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 677, in build_extensions
    build_ext.build_extensions(self)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
    self._build_extensions_serial()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
    self.build_extension(ext)
  File "/home/jzhao7/audio/build_tools/setup_helpers/extension.py", line 139, in build_extension
    _build_third_party()
  File "/home/jzhao7/audio/build_tools/setup_helpers/extension.py", line 113, in _build_third_party
    check=True,
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['cmake', '--build', '.']' returned non-zero exit status 2.

To confirm, here is what my 2 project lines look like:

ExternalProject_Add(libmad
  PREFIX ${CMAKE_CURRENT_SOURCE_DIR}
  DOWNLOAD_DIR ${ARCHIVE_DIR}
  URL https://downloads.sourceforge.net/project/mad/libmad/0.15.1b/libmad-0.15.1b.tar.gz
  URL_HASH SHA256=bbfac3ed6bfbc2823d3775ebb931087371e142bb0e9bb1bee51a76a6e0078690
  PATCH_COMMAND patch < ${CMAKE_CURRENT_SOURCE_DIR}/patch/libmad.patch && cp ${CMAKE_CURRENT_SOURCE_DIR}/patch/config.guess ${CMAKE_CURRENT_BINARY_DIR}/src/mad
/config.guess && cp ${CMAKE_CURRENT_SOURCE_DIR}/patch/config.sub ${CMAKE_CURRENT_BINARY_DIR}/src/mad/config.sub
  CONFIGURE_COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/src/libmad/configure ${COMMON_ARGS}
)
ExternalProject_Add(libmp3lame
  PREFIX ${CMAKE_CURRENT_SOURCE_DIR}
  DOWNLOAD_DIR ${ARCHIVE_DIR} 
  URL https://downloads.sourceforge.net/project/lame/lame/3.99/lame-3.99.5.tar.gz
  URL_HASH SHA256=24346b4158e4af3bd9f2e194bb23eb473c75fb7377011523353196b19b9a23ff
  PATCH_COMMAND cp ${CMAKE_CURRENT_SOURCE_DIR}/patch/config.guess ${CMAKE_CURRENT_SOURCE_DIR}/src/lame/config.guess && cp ${CMAKE_CURRENT_SOURCE_DIR}/patch/con
fig.sub ${CMAKE_CURRENT_SOURCE_DIR}/src/lame/config.sub
  CONFIGURE_COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/src/libmp3lame/configure ${COMMON_ARGS} --enable-nasm
)
mthrok commented 3 years ago

Some small detail was changed after 0.7. Can you try this?

I think if you check the content of third_party/src directory, you should see libmad/config.[gusee|sub] and libmp3lame/config.[guess|sub]. The PATCH_COMMAND should be overwriting them with the one from patch directory.

PATCH_COMMAND patch < ${CMAKE_CURRENT_SOURCE_DIR}/patch/libmad.patch && cp ${CMAKE_CURRENT_SOURCE_DIR}/patch/config.guess ${CMAKE_CURRENT_SOURCE_DIR}/src/libmad/config.guess && cp ${CMAKE_CURRENT_SOURCE_DIR}/patch/config.sub ${CMAKE_CURRENT_SOURCE_DIR}/src/libmad/config.sub
PATCH_COMMAND cp ${CMAKE_CURRENT_SOURCE_DIR}/patch/config.guess ${CMAKE_CURRENT_SOURCE_DIR}/src/libmp3lame/config.guess && cp ${CMAKE_CURRENT_SOURCE_DIR}/patch/config.sub ${CMAKE_CURRENT_SOURCE_DIR}/src/libmp3lame/config.sub
j4sonzhao commented 3 years ago

@mthrok

Thanks! It looks like these config.guess errors have been resolved. For future reference, the first line that needs to be changed in third_party/CMakeLists.txtshould read 

PATCH_COMMAND patch < ${CMAKE_CURRENT_SOURCE_DIR}/patch/libmad.patch && cp ${CMAKE_CURRENT_SOURCE_DIR}/patch/config.guess ${CMAKE_CURRENT_SOURCE_DIR}/src/mad/config.guess && cp ${CMAKE_CURRENT_SOURCE_DIR}/patch/config.sub ${CMAKE_CURRENT_SOURCE_DIR}/src/mad/config.sub

instead of 

PATCH_COMMAND patch < ${CMAKE_CURRENT_SOURCE_DIR}/patch/libmad.patch && cp ${CMAKE_CURRENT_SOURCE_DIR}/patch/config.guess ${CMAKE_CURRENT_BINARY_DIR}/src/mad/config.guess && cp ${CMAKE_CURRENT_SOURCE_DIR}/patch/config.sub ${CMAKE_CURRENT_BINARY_DIR}/src/mad/config.sub

I'm actually now getting a gcc related error, I think because I am running gcc version that's too high (8.3.1).

No CUDA runtime is found, using CUDA_HOME='/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real'
-- Building version 0.7.0a0+a853dff
running install
running bdist_egg
running egg_info
writing torchaudio.egg-info/PKG-INFO
writing dependency_links to torchaudio.egg-info/dependency_links.txt
writing requirements to torchaudio.egg-info/requires.txt
writing top-level names to torchaudio.egg-info/top_level.txt
reading manifest file 'torchaudio.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'torchaudio.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-ppc64le/egg
running install_lib
running build_py
copying torchaudio/version.py -> build/lib.linux-ppc64le-3.7/torchaudio
running build_ext
-- Configuring done
-- Generating done
-- Build files have been written to: /home/jzhao7/audio/third_party/build
[ 12%] Built target libogg
[ 25%] Built target libflac
[ 37%] Built target libopus
[ 50%] Built target opusfile
[ 62%] Built target libmp3lame
[ 64%] Performing patch step for 'libmad'
patching file configure
Reversed (or previously applied) patch detected!  Assume -R? [n] y
[ 65%] Performing configure step for 'libmad'
appending configuration tag "CXX" to libtool
appending configuration tag "F77" to libtool
[ 67%] Performing build step for 'libmad'
 gcc -DHAVE_CONFIG_H -I. -I/home/jzhao7/audio/third_party/src/libmad -I. -DFPM_PPC -Wall -g -O -fforce-mem -fforce-addr -fthread-jumps -fcse-follow-jumps -fcse-skip-blocks -fexpensive-optimizations -fregmove -fschedule-insns2 -c /home/jzhao7/audio/third_party/src/libmad/version.c  -fPIC -DPIC -o version.o
gcc: error: unrecognized command line option '-fforce-mem'; did you mean '-fforce-addr'?
gmake[5]: *** [Makefile:388: version.lo] Error 1
gmake[4]: *** [Makefile:424: all-recursive] Error 1
gmake[3]: *** [Makefile:249: all] Error 2
gmake[2]: *** [CMakeFiles/libmad.dir/build.make:86: ../src/libmad-stamp/libmad-build] Error 2
gmake[1]: *** [CMakeFiles/Makefile2:180: CMakeFiles/libmad.dir/all] Error 2
gmake: *** [Makefile:91: all] Error 2
Traceback (most recent call last):
  File "setup.py", line 89, in <module>
    zip_safe=False,
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/install.py", line 67, in run
    self.do_egg_install()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/install.py", line 109, in do_egg_install
    self.run_command('bdist_egg')
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 164, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 150, in call_command
    self.run_command(cmdname)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/install_lib.py", line 11, in run
    self.build()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/install_lib.py", line 107, in build
    self.run_command('build_ext')
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 79, in run
    _build_ext.run(self)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 677, in build_extensions
    build_ext.build_extensions(self)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
    self._build_extensions_serial()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
    self.build_extension(ext)
  File "/home/jzhao7/audio/build_tools/setup_helpers/extension.py", line 139, in build_extension
    _build_third_party()
  File "/home/jzhao7/audio/build_tools/setup_helpers/extension.py", line 113, in _build_third_party
    check=True,
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['cmake', '--build', '.']' returned non-zero exit status 2.

Following this post, I tried to see if I could change the makefile inside third_party/src/libmad but I didn't actually find this -fforce-mem tag in either of the makefiles. I also found this post, but I'm not entirely sure how I would link up this patch to the current installation process.

Help would be appreciated, thanks!

mthrok commented 3 years ago

Hi @j4sonzhao

I looked into it and indeed grep does not report any use of -fforce-mem flag. (note: configure.ac is not used during the build process)

% grep '\-fforce-mem' -r third_party/src
third_party/src/libmad/configure.ac:        optimize="$optimize -fforce-mem"

The patch you linked is already incorporated. You should see the same thing in third_party/patch/libmad.patch, and PATCH_COMMAND patch < ${CMAKE_CURRENT_SOURCE_DIR}/patch/libmad.patch is where the patch is applied. From the log message, it should be applied properly in your case.

[ 64%] Performing patch step for 'libmad'
patching file configure
Reversed (or previously applied) patch detected!  Assume -R? [n] y

Other possibility is environment variables. ./third_party/src/libmad/configure --help gives you the list of env vars that changes the behavior. However, IIRC, the builds happening inside of ExternalProject_Add do not automatically inherit environment variables from your current shell, so I actually doubt changing any of the following environment variable would help.

Some influential environment variables:
  CC          C compiler command
  CFLAGS      C compiler flags
  LDFLAGS     linker flags, e.g. -L<lib dir> if you have libraries in a
              nonstandard directory <lib dir>
  CPPFLAGS    C/C++ preprocessor flags, e.g. -I<include dir> if you have
              headers in a nonstandard directory <include dir>
  CCAS        assembler compiler command (defaults to CC)
  CCASFLAGS   assembler compiler flags (defaults to CFLAGS)
  CPP         C preprocessor
  CXX         C++ compiler command
  CXXFLAGS    C++ compiler flags
  CXXCPP      C++ preprocessor
  F77         Fortran 77 compiler command
  FFLAGS      Fortran 77 compiler flags

Use these variables to override the choices made by `configure' or to help
it to find libraries and programs with nonstandard names/locations.

Regardless of where the -fforce-mem flag is coming from, one way to override the actual CFLAGS during the compilation is to add CFLAGS=<flags you want to add> at the end of CONFIGURE_COMMAND.

To reset unset CFLAGS, you can try;

CONFIGURE_COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/src/libmad/configure ${COMMON_ARGS} CFLAGS=

This should apply to the other environment variables listed above.

j4sonzhao commented 3 years ago

@mthrok

Thanks! Changing that configure command worked and it looks like libsox installed successfully.

I reran python setup.py install, and some interesting things happened. I ran it 3 times, and on the first two times, it threw some c++ errors . But then, on the third time, it looks like it successfully installed. However, here is what I see if I import torchaudio:

image

Is this a problem?

Additionally, for reference I am posting the 3 stdout outputs I saw when running the command 3 times. Note that the first time, there were 6 commands run (i.e. [1/6] to [6/6]) and the second time there were 4 ([1/4], etc.), and the third time there was only one ([1/1]).

First:

Emitting ninja build file /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/6] c++ -MMD -MF /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox_utils.o.d -pthread -B /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jzhao7/audio -I/home/jzhao7/audio/third_party/install/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/TH -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/THC -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/include/python3.7m -c -c /home/jzhao7/audio/torchaudio/csrc/sox_utils.cpp -o /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox_utils.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_torchaudio -D_GLIBCXX_USE_CXX11_ABI=1 -std=gnu++14
FAILED: /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox_utils.o 
c++ -MMD -MF /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox_utils.o.d -pthread -B /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jzhao7/audio -I/home/jzhao7/audio/third_party/install/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/TH -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/THC -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/include/python3.7m -c -c /home/jzhao7/audio/torchaudio/csrc/sox_utils.cpp -o /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox_utils.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_torchaudio -D_GLIBCXX_USE_CXX11_ABI=1 -std=gnu++14
[2/6] c++ -MMD -MF /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox_effects_chain.o.d -pthread -B /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jzhao7/audio -I/home/jzhao7/audio/third_party/install/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/TH -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/THC -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/include/python3.7m -c -c /home/jzhao7/audio/torchaudio/csrc/sox_effects_chain.cpp -o /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox_effects_chain.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_torchaudio -D_GLIBCXX_USE_CXX11_ABI=1 -std=gnu++14
FAILED: /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox_effects_chain.o 
c++ -MMD -MF /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox_effects_chain.o.d -pthread -B /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jzhao7/audio -I/home/jzhao7/audio/third_party/install/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/TH -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/THC -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/include/python3.7m -c -c /home/jzhao7/audio/torchaudio/csrc/sox_effects_chain.cpp -o /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox_effects_chain.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_torchaudio -D_GLIBCXX_USE_CXX11_ABI=1 -std=gnu++14
[3/6] c++ -MMD -MF /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/register.o.d -pthread -B /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jzhao7/audio -I/home/jzhao7/audio/third_party/install/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/TH -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/THC -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/include/python3.7m -c -c /home/jzhao7/audio/torchaudio/csrc/register.cpp -o /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/register.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_torchaudio -D_GLIBCXX_USE_CXX11_ABI=1 -std=gnu++14
FAILED: /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/register.o 
c++ -MMD -MF /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/register.o.d -pthread -B /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jzhao7/audio -I/home/jzhao7/audio/third_party/install/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/TH -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/THC -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/include/python3.7m -c -c /home/jzhao7/audio/torchaudio/csrc/register.cpp -o /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/register.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_torchaudio -D_GLIBCXX_USE_CXX11_ABI=1 -std=gnu++14
7/site-packages/torch/include/torch/csrc/autograd/generated/variable_factories.h:12,
                 from /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/types.h:7,
                 from /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/torch/script.h:3,
                 from /home/jzhao7/audio/torchaudio/csrc/sox_effects.h:4,
                 from /home/jzhao7/audio/torchaudio/csrc/register.cpp:4:
/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/ATen/core/boxing/impl/make_boxed_from_unboxed_functor.h: In instFunc = std::vector<std::vector<std::__cxx11::basic_string<char> > >(); std::enable_if_t<c10::guts::is_function_type<FuncType_>::value, std::nullptr_t> = std::n
[4/6] c++ -MMD -MF /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox_effects.o.d -pthread -B /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jzhao7/audio -I/home/jzhao7/audio/third_party/install/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/TH -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/THC -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/include/python3.7m -c -c /home/jzhao7/audio/torchaudio/csrc/sox_effects.cpp -o /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox_effects.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_torchaudio -D_GLIBCXX_USE_CXX11_ABI=1 -std=gnu++14
[5/6] c++ -MMD -MF /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox_io.o.d -pthread -B /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jzhao7/audio -I/home/jzhao7/audio/third_party/install/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/TH -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/THC -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/include/python3.7m -c -c /home/jzhao7/audio/torchaudio/csrc/sox_io.cpp -o /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox_io.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_torchaudio -D_GLIBCXX_USE_CXX11_ABI=1 -std=gnu++14
[6/6] c++ -MMD -MF /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox.o.d -pthread -B /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jzhao7/audio -I/home/jzhao7/audio/third_party/install/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/TH -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/THC -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/include/python3.7m -c -c /home/jzhao7/audio/torchaudio/csrc/sox.cpp -o /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_torchaudio -D_GLIBCXX_USE_CXX11_ABI=1 -std=gnu++14
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1546, in _run_ninja_build
    env=env)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "setup.py", line 89, in <module>
    zip_safe=False,
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/install.py", line 67, in run
    self.do_egg_install()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/install.py", line 109, in do_egg_install
    self.run_command('bdist_egg')
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 164, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 150, in call_command
    self.run_command(cmdname)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/install_lib.py", line 11, in run
    self.build()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/install_lib.py", line 107, in build
    self.run_command('build_ext')
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 79, in run
    _build_ext.run(self)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 677, in build_extensions
    build_ext.build_extensions(self)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
    self._build_extensions_serial()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
    self.build_extension(ext)
  File "/home/jzhao7/audio/build_tools/setup_helpers/extension.py", line 140, in build_extension
    super().build_extension(ext)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 202, in build_extension
    _build_ext.build_extension(self, ext)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/build_ext.py", line 534, in build_extension
    depends=ext.depends)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 507, in unix_wrap_ninja_compile
    with_cuda=with_cuda)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1262, in _write_ninja_file_and_com
pile_objects
    error_prefix='Error compiling objects for extension')
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1562, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error compiling objects for extension

Second:

Emitting ninja build file /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/4] c++ -MMD -MF /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox.o.d -pthread -B /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jzhao7/audio -I/home/jzhao7/audio/third_party/install/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/TH -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/THC -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/include/python3.7m -c -c /home/jzhao7/audio/torchaudio/csrc/sox.cpp -o /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_torchaudio -D_GLIBCXX_USE_CXX11_ABI=1 -std=gnu++14
FAILED: /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox.o 
c++ -MMD -MF /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox.o.d -pthread -B /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jzhao7/audio -I/home/jzhao7/audio/third_party/install/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/TH -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/THC -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/include/python3.7m -c -c /home/jzhao7/audio/torchaudio/csrc/sox.cpp -o /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_torchaudio -D_GLIBCXX_USE_CXX11_ABI=1 -std=gnu++14
[2/4] c++ -MMD -MF /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox_effects_chain.o.d -pthread -B /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jzhao7/audio -I/home/jzhao7/audio/third_party/install/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/TH -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/THC -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/include/python3.7m -c -c /home/jzhao7/audio/torchaudio/csrc/sox_effects_chain.cpp -o /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox_effects_chain.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_torchaudio -D_GLIBCXX_USE_CXX11_ABI=1 -std=gnu++14
[3/4] c++ -MMD -MF /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox_utils.o.d -pthread -B /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jzhao7/audio -I/home/jzhao7/audio/third_party/install/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/TH -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/THC -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/include/python3.7m -c -c /home/jzhao7/audio/torchaudio/csrc/sox_utils.cpp -o /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox_utils.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_torchaudio -D_GLIBCXX_USE_CXX11_ABI=1 -std=gnu++14
[4/4] c++ -MMD -MF /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/register.o.d -pthread -B /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jzhao7/audio -I/home/jzhao7/audio/third_party/install/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/TH -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/THC -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/include/python3.7m -c -c /home/jzhao7/audio/torchaudio/csrc/register.cpp -o /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/register.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_torchaudio -D_GLIBCXX_USE_CXX11_ABI=1 -std=gnu++14
:vector<std::__cxx11::basic_string<char> > >; c10::Stack = std::vector<c10::IValue>; std::index_sequence<INDEX ...> = std::integer_sequence<long unsigned int>]10::impl::detail::WrapFunctionIntoRuntimeFunctor_<std::vector<std::__cxx11::basic_string<char> > (*)(), std::vector<std::__cxx11::basic_string<char> >, c10::guts::typelist::typelist<> >; bool AllowDeprecatedTypes = false; long unsigned int ...ivalue_arg_indices = {}; std::decay_t<typename c10::guts::infer_function_traits<Functor>::type::return_type> = std::vector<std::__cxx11::basic_string<char> >; c10::Stack = std::vector<c10::IValue>; std::index_sequence<INDEX ...> = std_functor_with_args_from_stack_(Functor*, c10::Stack*, std::index_sequence<INDEX ...>) [with Functor = c10::impl::detail::WrapFunctionIntoRuntimeFunctor_<void (*)(), void, c10::guts::typelist::typelist<> >; bool AllowDeprecatedTypes = false; long unsigned int ...ivalue_arg_indices = {}; std::decay_t<typename c10::guts::infer_function_traits<Functor>::type::return_type> = void; c10::Stack = std::vector<c10::IValue>; std::index_sequence<INDEX ...> = std::integer_sequence<long1.7real/lib/python3.7/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/install.py", line 67, in run
    self.do_egg_install()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/install.py", line 109, in do_egg_install
    self.run_command('bdist_egg')
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 164, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 150, in call_command
    self.run_command(cmdname)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/install_lib.py", line 11, in run
    self.build()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/install_lib.py", line 107, in build
FAILED: /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox.o 
    self.run_command('build_ext')
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 79, in run
    _build_ext.run(self)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 677, in build_extensions
    build_ext.build_extensions(self)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
    self._build_extensions_serial()
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
    self.build_extension(ext)
  File "/home/jzhao7/audio/build_tools/setup_helpers/extension.py", line 140, in build_extension
    super().build_extension(ext)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 202, in build_extension
    _build_ext.build_extension(self, ext)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/distutils/command/build_ext.py", line 534, in build_extension
    depends=ext.depends)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 507, in unix_wrap_ninja_compile
    with_cuda=with_cuda)
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1262, in _write_ninja_file_and_com
pile_objects
    error_prefix='Error compiling objects for extension')
  File "/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1562, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error compiling objects for extension

Third:

Emitting ninja build file /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/1] c++ -MMD -MF /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox.o.d -pthread -B /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jzhao7/audio -I/home/jzhao7/audio/third_party/install/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/TH -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torch/include/THC -I/nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/include/python3.7m -c -c /home/jzhao7/audio/torchaudio/csrc/sox.cpp -o /home/jzhao7/audio/build/temp.linux-ppc64le-3.7/home/jzhao7/audio/torchaudio/csrc/sox.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_torchaudio -D_GLIBCXX_USE_CXX11_ABI=1 -std=gnu++14
python -o build/lib.linux-ppc64le-3.7/torchaudio/_torchaudio.so -O3
creating build/bdist.linux-ppc64le
creating build/bdist.linux-ppc64le/egg
creating build/bdist.linux-ppc64le/egg/torchaudio
creating build/bdist.linux-ppc64le/egg/torchaudio/compliance
copying build/lib.linux-ppc64le-3.7/torchaudio/compliance/kaldi.py -> build/bdist.linux-ppc64le/egg/torchaudio/compliance
copying build/lib.linux-ppc64le-3.7/torchaudio/compliance/__init__.py -> build/bdist.linux-ppc64le/egg/torchaudio/compliance
copying build/lib.linux-ppc64le-3.7/torchaudio/transforms.py -> build/bdist.linux-ppc64le/egg/torchaudio
creating build/bdist.linux-ppc64le/egg/torchaudio/_internal
copying build/lib.linux-ppc64le-3.7/torchaudio/_internal/misc_ops.py -> build/bdist.linux-ppc64le/egg/torchaudio/_internal
copying build/lib.linux-ppc64le-3.7/torchaudio/_internal/module_utils.py -> build/bdist.linux-ppc64le/egg/torchaudio/_internal
copying build/lib.linux-ppc64le-3.7/torchaudio/_internal/__init__.py -> build/bdist.linux-ppc64le/egg/torchaudio/_internal
creating build/bdist.linux-ppc64le/egg/torchaudio/sox_effects
copying build/lib.linux-ppc64le-3.7/torchaudio/sox_effects/__init__.py -> build/bdist.linux-ppc64le/egg/torchaudio/sox_effects
copying build/lib.linux-ppc64le-3.7/torchaudio/sox_effects/sox_effects.py -> build/bdist.linux-ppc64le/egg/torchaudio/sox_effects
creating build/bdist.linux-ppc64le/egg/torchaudio/datasets
copying build/lib.linux-ppc64le-3.7/torchaudio/datasets/gtzan.py -> build/bdist.linux-ppc64le/egg/torchaudio/datasets
copying build/lib.linux-ppc64le-3.7/torchaudio/datasets/librispeech.py -> build/bdist.linux-ppc64le/egg/torchaudio/datasets
copying build/lib.linux-ppc64le-3.7/torchaudio/datasets/tedlium.py -> build/bdist.linux-ppc64le/egg/torchaudio/datasets
copying build/lib.linux-ppc64le-3.7/torchaudio/datasets/libritts.py -> build/bdist.linux-ppc64le/egg/torchaudio/datasets
copying build/lib.linux-ppc64le-3.7/torchaudio/datasets/__init__.py -> build/bdist.linux-ppc64le/egg/torchaudio/datasets
copying build/lib.linux-ppc64le-3.7/torchaudio/datasets/utils.py -> build/bdist.linux-ppc64le/egg/torchaudio/datasets
copying build/lib.linux-ppc64le-3.7/torchaudio/datasets/ljspeech.py -> build/bdist.linux-ppc64le/egg/torchaudio/datasets
copying build/lib.linux-ppc64le-3.7/torchaudio/datasets/commonvoice.py -> build/bdist.linux-ppc64le/egg/torchaudio/datasets
copying build/lib.linux-ppc64le-3.7/torchaudio/datasets/vctk.py -> build/bdist.linux-ppc64le/egg/torchaudio/datasets
copying build/lib.linux-ppc64le-3.7/torchaudio/datasets/yesno.py -> build/bdist.linux-ppc64le/egg/torchaudio/datasets
copying build/lib.linux-ppc64le-3.7/torchaudio/datasets/speechcommands.py -> build/bdist.linux-ppc64le/egg/torchaudio/datasets
copying build/lib.linux-ppc64le-3.7/torchaudio/datasets/cmuarctic.py -> build/bdist.linux-ppc64le/egg/torchaudio/datasets
copying build/lib.linux-ppc64le-3.7/torchaudio/_torchaudio.so -> build/bdist.linux-ppc64le/egg/torchaudio
copying build/lib.linux-ppc64le-3.7/torchaudio/__init__.py -> build/bdist.linux-ppc64le/egg/torchaudio
copying build/lib.linux-ppc64le-3.7/torchaudio/kaldi_io.py -> build/bdist.linux-ppc64le/egg/torchaudio
creating build/bdist.linux-ppc64le/egg/torchaudio/utils
copying build/lib.linux-ppc64le-3.7/torchaudio/utils/__init__.py -> build/bdist.linux-ppc64le/egg/torchaudio/utils
copying build/lib.linux-ppc64le-3.7/torchaudio/utils/sox_utils.py -> build/bdist.linux-ppc64le/egg/torchaudio/utils
creating build/bdist.linux-ppc64le/egg/torchaudio/models
copying build/lib.linux-ppc64le-3.7/torchaudio/models/wav2letter.py -> build/bdist.linux-ppc64le/egg/torchaudio/models
copying build/lib.linux-ppc64le-3.7/torchaudio/models/__init__.py -> build/bdist.linux-ppc64le/egg/torchaudio/models
copying build/lib.linux-ppc64le-3.7/torchaudio/models/conv_tasnet.py -> build/bdist.linux-ppc64le/egg/torchaudio/models
copying build/lib.linux-ppc64le-3.7/torchaudio/models/wavernn.py -> build/bdist.linux-ppc64le/egg/torchaudio/models
creating build/bdist.linux-ppc64le/egg/torchaudio/extension
copying build/lib.linux-ppc64le-3.7/torchaudio/extension/extension.py -> build/bdist.linux-ppc64le/egg/torchaudio/extension
copying build/lib.linux-ppc64le-3.7/torchaudio/extension/__init__.py -> build/bdist.linux-ppc64le/egg/torchaudio/extension
copying build/lib.linux-ppc64le-3.7/torchaudio/version.py -> build/bdist.linux-ppc64le/egg/torchaudio
creating build/bdist.linux-ppc64le/egg/torchaudio/backend
copying build/lib.linux-ppc64le-3.7/torchaudio/backend/sox_backend.py -> build/bdist.linux-ppc64le/egg/torchaudio/backend
copying build/lib.linux-ppc64le-3.7/torchaudio/backend/__init__.py -> build/bdist.linux-ppc64le/egg/torchaudio/backend
copying build/lib.linux-ppc64le-3.7/torchaudio/backend/utils.py -> build/bdist.linux-ppc64le/egg/torchaudio/backend
copying build/lib.linux-ppc64le-3.7/torchaudio/backend/sox_io_backend.py -> build/bdist.linux-ppc64le/egg/torchaudio/backend
copying build/lib.linux-ppc64le-3.7/torchaudio/backend/common.py -> build/bdist.linux-ppc64le/egg/torchaudio/backend
copying build/lib.linux-ppc64le-3.7/torchaudio/backend/soundfile_backend.py -> build/bdist.linux-ppc64le/egg/torchaudio/backend
copying build/lib.linux-ppc64le-3.7/torchaudio/backend/no_backend.py -> build/bdist.linux-ppc64le/egg/torchaudio/backend
copying build/lib.linux-ppc64le-3.7/torchaudio/backend/_soundfile_backend.py -> build/bdist.linux-ppc64le/egg/torchaudio/backend
copying build/lib.linux-ppc64le-3.7/torchaudio/functional.py -> build/bdist.linux-ppc64le/egg/torchaudio
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/compliance/kaldi.py to kaldi.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/compliance/__init__.py to __init__.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/transforms.py to transforms.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/_internal/misc_ops.py to misc_ops.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/_internal/module_utils.py to module_utils.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/_internal/__init__.py to __init__.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/sox_effects/__init__.py to __init__.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/sox_effects/sox_effects.py to sox_effects.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/datasets/gtzan.py to gtzan.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/datasets/librispeech.py to librispeech.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/datasets/tedlium.py to tedlium.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/datasets/libritts.py to libritts.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/datasets/__init__.py to __init__.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/datasets/utils.py to utils.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/datasets/ljspeech.py to ljspeech.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/datasets/commonvoice.py to commonvoice.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/datasets/vctk.py to vctk.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/datasets/yesno.py to yesno.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/datasets/speechcommands.py to speechcommands.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/datasets/cmuarctic.py to cmuarctic.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/__init__.py to __init__.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/kaldi_io.py to kaldi_io.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/utils/__init__.py to __init__.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/utils/sox_utils.py to sox_utils.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/models/wav2letter.py to wav2letter.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/models/__init__.py to __init__.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/models/conv_tasnet.py to conv_tasnet.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/models/wavernn.py to wavernn.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/extension/extension.py to extension.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/extension/__init__.py to __init__.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/version.py to version.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/backend/sox_backend.py to sox_backend.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/backend/__init__.py to __init__.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/backend/utils.py to utils.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/backend/sox_io_backend.py to sox_io_backend.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/backend/common.py to common.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/backend/soundfile_backend.py to soundfile_backend.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/backend/no_backend.py to no_backend.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/backend/_soundfile_backend.py to _soundfile_backend.cpython-37.pyc
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/functional.py to functional.cpython-37.pyc
creating stub loader for torchaudio/_torchaudio.so
byte-compiling build/bdist.linux-ppc64le/egg/torchaudio/_torchaudio.py to _torchaudio.cpython-37.pyc
creating build/bdist.linux-ppc64le/egg/EGG-INFO
copying torchaudio.egg-info/PKG-INFO -> build/bdist.linux-ppc64le/egg/EGG-INFO
copying torchaudio.egg-info/SOURCES.txt -> build/bdist.linux-ppc64le/egg/EGG-INFO
copying torchaudio.egg-info/dependency_links.txt -> build/bdist.linux-ppc64le/egg/EGG-INFO
copying torchaudio.egg-info/not-zip-safe -> build/bdist.linux-ppc64le/egg/EGG-INFO
copying torchaudio.egg-info/requires.txt -> build/bdist.linux-ppc64le/egg/EGG-INFO
copying torchaudio.egg-info/top_level.txt -> build/bdist.linux-ppc64le/egg/EGG-INFO
writing build/bdist.linux-ppc64le/egg/EGG-INFO/native_libs.txt
creating dist
creating 'dist/torchaudio-0.7.0a0+a853dff-py3.7-linux-ppc64le.egg' and adding 'build/bdist.linux-ppc64le/egg' to it
removing 'build/bdist.linux-ppc64le/egg' (and everything under it)
Processing torchaudio-0.7.0a0+a853dff-py3.7-linux-ppc64le.egg
creating /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torchaudio-0.7.0a0+a853dff-py3.7-linux-ppc64le.egg
Extracting torchaudio-0.7.0a0+a853dff-py3.7-linux-ppc64le.egg to /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages
Adding torchaudio 0.7.0a0+a853dff to easy-install.pth file
Installed /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages/torchaudio-0.7.0a0+a853dff-py3.7-linux-ppc64le.egg
Processing dependencies for torchaudio==0.7.0a0+a853dff
Searching for torch==1.7.1
Best match: torch 1.7.1
Adding torch 1.7.1 to easy-install.pth file
Installing convert-caffe2-to-onnx script to /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/bin
Installing convert-onnx-to-caffe2 script to /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/bin
Using /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages
Searching for numpy==1.19.2
Best match: numpy 1.19.2
Adding numpy 1.19.2 to easy-install.pth file
Installing f2py script to /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/bin
Installing f2py3 script to /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/bin
Installing f2py3.7 script to /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/bin
Using /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages
Searching for typing-extensions==3.7.4.3
Best match: typing-extensions 3.7.4.3
Adding typing-extensions 3.7.4.3 to easy-install.pth file
Using /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages
Finished processing dependencies for torchaudio==0.7.0a0+a853dff

Help would be appreciated, thanks!

j4sonzhao commented 3 years ago

@mthrok

To provide more information, when I type conda list I see torchaudio as

torchaudio 0.7.0a0+a853dff pypi_0 pypi

torchaudio is a dependency for some repositories that I would like to install, for example this one. However, when I run that repository's installation procedure, which just consists of pip install -e ./, I get the error 

ERROR: Could not find a version that satisfies the requirement torchaudio>=0.7.0 (from s3prl)
ERROR: No matching distribution found for torchaudio>=0.7.0

which leads me to believe that torchaudio was not installed properly.

mthrok commented 3 years ago

I reran python setup.py install, and some interesting things happened. I ran it 3 times, and on the first two times, it threw some c++ errors . But then, on the third time, it looks like it successfully installed. However, here is what I see if I import torchaudio:

image

Is this a problem?

@j4sonzhao

Regarding the build process behavior, this is expected. The build process (Ninja) attempts to compile multiple objects at the same time, then it caches the successfully compiled objects until the compilation configuration is changed later. I guess what happened is that in the early attempts, some of compile processes hit out-of-memory and reported error, but some other compilation process were successful and the resulting object was cached. And, in the subsequent run, they are not compiled. I think setting environment variable MAX_JOBS=1 will avoid the OOM issue.

Now about the warning when import torchaudio, this happens because import torchaudio is executed in the root repository of torchaudio. Looking at the build process log, you invoked python setup.py install, and it successfully installed torchaudio in /nobackup/users/jzhao7/anaconda3/envs/pytorch1.7real/lib/python3.7/site-packages. However, this installation is shadowed by the source torchaudio directory when you run Python process in root directory of torchaudio repository. It is our intent that torchaudio library code is usable without building the extension, so Python had no problem picking up the source torchaudio and the mechanism that initializes the extension and audio backend detected that the extension is not installed there, thus issued the warning. Long story short, since your goal is to use s3prl and not developing torchaudio, I think simply moving out of the torchaudio source directory and do import torchaudio, it should work. (also try print(torchaudio.__version__))

The version number (0.7.0a0+a853dff) is slightly incorrect (it is supposed to be 0.7.2), but this is on us. We set the version number for binary distribution in a different manner and forgot to update in the source. a853dff indicates that the version conda recognizes is indeed the one you compiled manually.

So, can you retry the installation with BUILD_VERSION=0.7.2 BUILD_SOX=1 MAX_JOBS=1 python setup.py clean install?

j4sonzhao commented 3 years ago

thank you so much! it looks like everything is running smoothly on my end now, and I really appreciate your help walking me through this setup!