justanhduc / neuralnet-pytorch

A high level framework for general purpose neural networks in Pytorch.
https://neuralnet-pytorch.readthedocs.io
MIT License
27 stars 6 forks source link

Installation error #3

Closed ZOUKaifeng closed 4 years ago

ZOUKaifeng commented 4 years ago

Hi, I am trying to install branch fancy by pip install git+git://github.com/justanhduc/neuralnet-pytorch.git@fancy But I failed to compile. It's because of the version of cuda?

FAILED: /tmp/pip-req-build-_phlkcei/build/temp.linux-x86_64-3.6/neuralnet_pytorch/extensions/cuda/emd_c/emd_kernel.o
    /usr/local/cuda/bin/nvcc -I/home/zou/.conda/envs/img/lib/python3.6/site-packages/torch/include -I/home/zou/.conda/envs/img/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/home/zou/.conda/envs/img/lib/python3.6/site-packages/torch/include/TH -I/home/zou/.conda/envs/img/lib/python3.6/site-packages/torch/include/THC -I/usr/local/cuda/include -I/home/zou/.conda/envs/img/include/python3.6m -c -c /tmp/pip-req-build-_phlkcei/neuralnet_pytorch/extensions/cuda/emd_c/emd_kernel.cu -o /tmp/pip-req-build-_phlkcei/build/temp.linux-x86_64-3.6/neuralnet_pytorch/extensions/cuda/emd_c/emd_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=emd_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=sm_75 -std=c++14
    /tmp/pip-req-build-_phlkcei/neuralnet_pytorch/extensions/cuda/emd_c/emd_kernel.cu(181): error: identifier "AT_CHECK" is undefined

    /tmp/pip-req-build-_phlkcei/neuralnet_pytorch/extensions/cuda/emd_c/emd_kernel.cu(268): error: identifier "AT_CHECK" is undefined

    /tmp/pip-req-build-_phlkcei/neuralnet_pytorch/extensions/cuda/emd_c/emd_kernel.cu(385): error: identifier "AT_CHECK" is undefined

    3 errors detected in the compilation of "/tmp/tmpxft_00001d1c_00000000-6_emd_kernel.cpp1.ii".
    [2/2] c++ -MMD -MF /tmp/pip-req-build-_phlkcei/build/temp.linux-x86_64-3.6/neuralnet_pytorch/extensions/cuda/emd_c/emd.o.d -pthread -B /home/zou/.conda/envs/img/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/zou/.conda/envs/img/lib/python3.6/site-packages/torch/include -I/home/zou/.conda/envs/img/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/home/zou/.conda/envs/img/lib/python3.6/site-packages/torch/include/TH -I/home/zou/.conda/envs/img/lib/python3.6/site-packages/torch/include/THC -I/usr/local/cuda/include -I/home/zou/.conda/envs/img/include/python3.6m -c -c /tmp/pip-req-build-_phlkcei/neuralnet_pytorch/extensions/cuda/emd_c/emd.cpp -o /tmp/pip-req-build-_phlkcei/build/temp.linux-x86_64-3.6/neuralnet_pytorch/extensions/cuda/emd_c/emd.o -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=emd_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
    cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
    ninja: build stopped: subcommand failed.
    Traceback (most recent call last):
      File "/home/zou/.conda/envs/img/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1400, in _run_ninja_build
        check=True)
      File "/home/zou/.conda/envs/img/lib/python3.6/subprocess.py", line 438, in run
        output=stdout, stderr=stderr)
    subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-req-build-_phlkcei/setup.py", line 82, in <module>
        setup_package()
      File "/tmp/pip-req-build-_phlkcei/setup.py", line 76, in setup_package
        'Source': 'https://github.com/justanhduc/neuralnet-pytorch',
      File "/home/zou/.conda/envs/img/lib/python3.6/site-packages/setuptools/__init__.py", line 144, in setup
        return distutils.core.setup(**attrs)
      File "/home/zou/.conda/envs/img/lib/python3.6/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/home/zou/.conda/envs/img/lib/python3.6/distutils/dist.py", line 955, in run_commands
        self.run_command(cmd)
      File "/home/zou/.conda/envs/img/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/home/zou/.conda/envs/img/lib/python3.6/site-packages/setuptools/command/install.py", line 61, in run
        return orig.install.run(self)
      File "/home/zou/.conda/envs/img/lib/python3.6/distutils/command/install.py", line 545, in run
        self.run_command('build')
      File "/home/zou/.conda/envs/img/lib/python3.6/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/home/zou/.conda/envs/img/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/home/zou/.conda/envs/img/lib/python3.6/distutils/command/build.py", line 135, in run
        self.run_command(cmd_name)
      File "/home/zou/.conda/envs/img/lib/python3.6/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/home/zou/.conda/envs/img/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/home/zou/.conda/envs/img/lib/python3.6/site-packages/setuptools/command/build_ext.py", line 87, in run
        _build_ext.run(self)
      File "/home/zou/.conda/envs/img/lib/python3.6/distutils/command/build_ext.py", line 339, in run
        self.build_extensions()
      File "/home/zou/.conda/envs/img/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 580, in build_extensions
        build_ext.build_extensions(self)
      File "/home/zou/.conda/envs/img/lib/python3.6/distutils/command/build_ext.py", line 448, in build_extensions
        self._build_extensions_serial()
      File "/home/zou/.conda/envs/img/lib/python3.6/distutils/command/build_ext.py", line 473, in _build_extensions_serial
        self.build_extension(ext)
      File "/home/zou/.conda/envs/img/lib/python3.6/site-packages/setuptools/command/build_ext.py", line 208, in build_extension
        _build_ext.build_extension(self, ext)
      File "/home/zou/.conda/envs/img/lib/python3.6/distutils/command/build_ext.py", line 533, in build_extension
        depends=ext.depends)
      File "/home/zou/.conda/envs/img/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 423, in unix_wrap_ninja_compile
        with_cuda=with_cuda)
      File "/home/zou/.conda/envs/img/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1140, in _write_ninja_file_and_compile_objects
        error_prefix='Error compiling objects for extension')
      File "/home/zou/.conda/envs/img/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1413, in _run_ninja_build
        raise RuntimeError(message)
    RuntimeError: Error compiling objects for extension
    ----------------------------------------
ERROR: Command errored out with exit status 1: /home/zou/.conda/envs/img/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-_phlkcei/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-_phlkcei/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-pnydvxnz/install-record.txt --single-version-externally-managed --compile --install-headers /home/zou/.conda/envs/img/include/python3.6m/neuralnet-pytorch Check the logs for full command output.
justanhduc commented 4 years ago

Hi. Could you please show the log? Here, it doesn't show what the real errors are. Also, I have never tried ninja build. What is your pytorch version? Could you please try distutils following here (scroll down to the utils.cpp_extensions section)?

ZOUKaifeng commented 4 years ago

Hi, thanks for your reponse. I use following code.

 cmdclass={'clean': clean,
 'build_ext': BuildExtension.with_options(use_ninja=False)},

Then I got:

running build
running build_py
UPDATING build/lib.linux-x86_64-3.6/neuralnet_pytorch/_version.py
set build/lib.linux-x86_64-3.6/neuralnet_pytorch/_version.py to '1.0.0+fancy.22.gf1bcfa1.dirty'
running build_ext
building 'emd_cuda' extension
gcc -pthread -B /home/zou/.conda/envs/img/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/zou/.conda/envs/img/lib/python3.6/site-packages/torch/include -I/home/zou/.conda/envs/img/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/home/zou/.conda/envs/img/lib/python3.6/site-packages/torch/include/TH -I/home/zou/.conda/envs/img/lib/python3.6/site-packages/torch/include/THC -I/usr/local/cuda/include -I/home/zou/.conda/envs/img/include/python3.6m -c neuralnet_pytorch/extensions/cuda/emd_c/emd.cpp -o build/temp.linux-x86_64-3.6/neuralnet_pytorch/extensions/cuda/emd_c/emd.o -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=emd_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
/usr/local/cuda/bin/nvcc -I/home/zou/.conda/envs/img/lib/python3.6/site-packages/torch/include -I/home/zou/.conda/envs/img/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/home/zou/.conda/envs/img/lib/python3.6/site-packages/torch/include/TH -I/home/zou/.conda/envs/img/lib/python3.6/site-packages/torch/include/THC -I/usr/local/cuda/include -I/home/zou/.conda/envs/img/include/python3.6m -c neuralnet_pytorch/extensions/cuda/emd_c/emd_kernel.cu -o build/temp.linux-x86_64-3.6/neuralnet_pytorch/extensions/cuda/emd_c/emd_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=emd_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=sm_75 -std=c++14
neuralnet_pytorch/extensions/cuda/emd_c/emd_kernel.cu(181): error: identifier "AT_CHECK" is undefined

neuralnet_pytorch/extensions/cuda/emd_c/emd_kernel.cu(268): error: identifier "AT_CHECK" is undefined

neuralnet_pytorch/extensions/cuda/emd_c/emd_kernel.cu(385): error: identifier "AT_CHECK" is undefined

3 errors detected in the compilation of "/tmp/tmpxft_00000b6f_00000000-6_emd_kernel.cpp1.ii".
error: command '/usr/local/cuda/bin/nvcc' failed with exit status 1

My env ubuntu server 18.04 pytorch 1.5 cuda 10.2 python 3.6

justanhduc commented 4 years ago

I haven't tried pytorch 1.5 and cuda 10.2 yet but the reason is this. Could you please change AT_CHECK to TORCH_CHECK and compile again?

ZOUKaifeng commented 4 years ago

OK! It works! Thank you!