Tramac / awesome-semantic-segmentation-pytorch

Semantic Segmentation on PyTorch (include FCN, PSPNet, Deeplabv3, Deeplabv3+, DANet, DenseASPP, BiSeNet, EncNet, DUNet, ICNet, ENet, OCNet, CCNet, PSANet, CGNet, ESPNet, LEDNet, DFANet)
Apache License 2.0
2.8k stars 581 forks source link

some problems about "python setup.py build develop" #131

Closed NoHateAnymore closed 3 years ago

NoHateAnymore commented 4 years ago

I type "python setup.py build develop" and there are some errors I can not solve them.

There is error information:

   ......
4 errors detected in the compilation of 
"C:/Users/35093/AppData/Local/Temp/tmpxft_00002d74_00000000-10_syncbn_cuda.cpp1.ii".
syncbn_cuda.cu
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "D:\anaconda3\envs\torch1\lib\site-packages\torch\utils\cpp_extension.py", line 1400, in _run_ninja_build
    check=True)
  File "D:\anaconda3\envs\torch1\lib\subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "setup.py", line 55, in <module>
    cmdclass={"build_ext": BuildExtension}
  File "D:\anaconda3\envs\torch1\lib\site-packages\setuptools\__init__.py", line 144, in setup
    return distutils.core.setup(**attrs)
  File "D:\anaconda3\envs\torch1\lib\distutils\core.py", line 148, in setup
    dist.run_commands()
  File "D:\anaconda3\envs\torch1\lib\distutils\dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "D:\anaconda3\envs\torch1\lib\distutils\dist.py", line 985, in run_command
    cmd_obj.run()
  File "D:\anaconda3\envs\torch1\lib\distutils\command\build.py", line 135, in run
    self.run_command(cmd_name)
  File "D:\anaconda3\envs\torch1\lib\distutils\cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "D:\anaconda3\envs\torch1\lib\distutils\dist.py", line 985, in run_command
    cmd_obj.run()
  File "D:\anaconda3\envs\torch1\lib\site-packages\setuptools\command\build_ext.py", line 87, in run
    _build_ext.run(self)
  File "D:\anaconda3\envs\torch1\lib\distutils\command\build_ext.py", line 340, in run
    self.build_extensions()
  File "D:\anaconda3\envs\torch1\lib\site-packages\torch\utils\cpp_extension.py", line 580, in build_extensions
    build_ext.build_extensions(self)
  File "D:\anaconda3\envs\torch1\lib\distutils\command\build_ext.py", line 449, in build_extensions
    self._build_extensions_serial()
  File "D:\anaconda3\envs\torch1\lib\distutils\command\build_ext.py", line 474, in _build_extensions_serial
    self.build_extension(ext)
  File "D:\anaconda3\envs\torch1\lib\site-packages\setuptools\command\build_ext.py", line 208, in build_extension
    _build_ext.build_extension(self, ext)
  File "D:\anaconda3\envs\torch1\lib\distutils\command\build_ext.py", line 534, in build_extension
    depends=ext.depends)
  File "D:\anaconda3\envs\torch1\lib\site-packages\torch\utils\cpp_extension.py", line 562, in win_wrap_ninja_compile
    with_cuda=with_cuda)
  File "D:\anaconda3\envs\torch1\lib\site-packages\torch\utils\cpp_extension.py", line 1140, in _write_ninja_file_and_compile_objects
    error_prefix='Error compiling objects for extension')
  File "D:\anaconda3\envs\torch1\lib\site-packages\torch\utils\cpp_extension.py", line 1413, in _run_ninja_build
    raise RuntimeError(message)
RuntimeError: Error compiling objects for extension

my pc System: Win10 Python: 3.7.7 pytorch: 1.5.0 cudatoolkit : 10.1.243 cudnn: 7.6.5

Can anyone help me? Thanks~

Zhang-O commented 4 years ago

Maybe you should try it in linux sys. I believe the code is build for linux,not for windows.

xuyingheng commented 4 years ago

man,I have same problem in ubuntu

HardworkingLittlequ commented 3 years ago

I have the same problem running build running build_ext building '._C' extension Emitting ninja build file /home/he/awesome-semantic-segmentation-pytorch-master/core/nn/build/temp.linux-x86_64-3.7/build.ninja... Compiling objects... Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N) [1/3] /usr/local/cuda-8.0/bin/nvcc -DWITH_CUDA -I/home/he/awesome-semantic-segmentation-pytorch-master/core/nn/csrc -I/home/he/anaconda3/lib/python3.7/site-packages/torch/include -I/home/he/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/he/anaconda3/lib/python3.7/site-packages/torch/include/TH -I/home/he/anaconda3/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda-8.0/include -I/home/he/anaconda3/include/python3.7m -c -c /home/he/awesome-semantic-segmentation-pytorch-master/core/nn/csrc/cuda/syncbn_cuda.cu -o /home/he/awesome-semantic-segmentation-pytorch-master/core/nn/build/temp.linux-x86_64-3.7/home/he/awesome-semantic-segmentation-pytorch-master/core/nn/csrc/cuda/syncbn_cuda.o -DCUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_61,code=sm_61 -std=c++14 FAILED: /home/he/awesome-semantic-segmentation-pytorch-master/core/nn/build/temp.linux-x86_64-3.7/home/he/awesome-semantic-segmentation-pytorch-master/core/nn/csrc/cuda/syncbn_cuda.o /usr/local/cuda-8.0/bin/nvcc -DWITH_CUDA -I/home/he/awesome-semantic-segmentation-pytorch-master/core/nn/csrc -I/home/he/anaconda3/lib/python3.7/site-packages/torch/include -I/home/he/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/he/anaconda3/lib/python3.7/site-packages/torch/include/TH -I/home/he/anaconda3/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda-8.0/include -I/home/he/anaconda3/include/python3.7m -c -c /home/he/awesome-semantic-segmentation-pytorch-master/core/nn/csrc/cuda/syncbn_cuda.cu -o /home/he/awesome-semantic-segmentation-pytorch-master/core/nn/build/temp.linux-x86_64-3.7/home/he/awesome-semantic-segmentation-pytorch-master/core/nn/csrc/cuda/syncbn_cuda.o -DCUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_61,code=sm_61 -std=c++14 nvcc fatal : Value 'c++14' is not defined for option 'std' [2/3] /usr/local/cuda-8.0/bin/nvcc -DWITH_CUDA -I/home/he/awesome-semantic-segmentation-pytorch-master/core/nn/csrc -I/home/he/anaconda3/lib/python3.7/site-packages/torch/include -I/home/he/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/he/anaconda3/lib/python3.7/site-packages/torch/include/TH -I/home/he/anaconda3/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda-8.0/include -I/home/he/anaconda3/include/python3.7m -c -c /home/he/awesome-semantic-segmentation-pytorch-master/core/nn/csrc/cuda/ca_cuda.cu -o /home/he/awesome-semantic-segmentation-pytorch-master/core/nn/build/temp.linux-x86_64-3.7/home/he/awesome-semantic-segmentation-pytorch-master/core/nn/csrc/cuda/ca_cuda.o -D__CUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_61,code=sm_61 -std=c++14 FAILED: /home/he/awesome-semantic-segmentation-pytorch-master/core/nn/build/temp.linux-x86_64-3.7/home/he/awesome-semantic-segmentation-pytorch-master/core/nn/csrc/cuda/ca_cuda.o /usr/local/cuda-8.0/bin/nvcc -DWITH_CUDA -I/home/he/awesome-semantic-segmentation-pytorch-master/core/nn/csrc -I/home/he/anaconda3/lib/python3.7/site-packages/torch/include -I/home/he/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/he/anaconda3/lib/python3.7/site-packages/torch/include/TH -I/home/he/anaconda3/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda-8.0/include -I/home/he/anaconda3/include/python3.7m -c -c /home/he/awesome-semantic-segmentation-pytorch-master/core/nn/csrc/cuda/ca_cuda.cu -o /home/he/awesome-semantic-segmentation-pytorch-master/core/nn/build/temp.linux-x86_64-3.7/home/he/awesome-semantic-segmentation-pytorch-master/core/nn/csrc/cuda/ca_cuda.o -DCUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_61,code=sm_61 -std=c++14 nvcc fatal : Value 'c++14' is not defined for option 'std' [3/3] /usr/local/cuda-8.0/bin/nvcc -DWITH_CUDA -I/home/he/awesome-semantic-segmentation-pytorch-master/core/nn/csrc -I/home/he/anaconda3/lib/python3.7/site-packages/torch/include -I/home/he/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/he/anaconda3/lib/python3.7/site-packages/torch/include/TH -I/home/he/anaconda3/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda-8.0/include -I/home/he/anaconda3/include/python3.7m -c -c /home/he/awesome-semantic-segmentation-pytorch-master/core/nn/csrc/cuda/psa_cuda.cu -o /home/he/awesome-semantic-segmentation-pytorch-master/core/nn/build/temp.linux-x86_64-3.7/home/he/awesome-semantic-segmentation-pytorch-master/core/nn/csrc/cuda/psa_cuda.o -DCUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_61,code=sm_61 -std=c++14 FAILED: /home/he/awesome-semantic-segmentation-pytorch-master/core/nn/build/temp.linux-x86_64-3.7/home/he/awesome-semantic-segmentation-pytorch-master/core/nn/csrc/cuda/psa_cuda.o /usr/local/cuda-8.0/bin/nvcc -DWITH_CUDA -I/home/he/awesome-semantic-segmentation-pytorch-master/core/nn/csrc -I/home/he/anaconda3/lib/python3.7/site-packages/torch/include -I/home/he/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/he/anaconda3/lib/python3.7/site-packages/torch/include/TH -I/home/he/anaconda3/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda-8.0/include -I/home/he/anaconda3/include/python3.7m -c -c /home/he/awesome-semantic-segmentation-pytorch-master/core/nn/csrc/cuda/psa_cuda.cu -o /home/he/awesome-semantic-segmentation-pytorch-master/core/nn/build/temp.linux-x86_64-3.7/home/he/awesome-semantic-segmentation-pytorch-master/core/nn/csrc/cuda/psa_cuda.o -D__CUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_61,code=sm_61 -std=c++14 nvcc fatal : Value 'c++14' is not defined for option 'std' ninja: build stopped: subcommand failed. Traceback (most recent call last): File "/home/he/anaconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1515, in _run_ninja_build env=env) File "/home/he/anaconda3/lib/python3.7/subprocess.py", line 512, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "setup.py", line 55, in cmdclass={"build_ext": BuildExtension} File "/home/he/anaconda3/lib/python3.7/site-packages/setuptools/init.py", line 165, in setup return distutils.core.setup(**attrs) File "/home/he/anaconda3/lib/python3.7/distutils/core.py", line 148, in setup dist.run_commands() File "/home/he/anaconda3/lib/python3.7/distutils/dist.py", line 966, in run_commands self.run_command(cmd) File "/home/he/anaconda3/lib/python3.7/distutils/dist.py", line 985, in run_command cmd_obj.run() File "/home/he/anaconda3/lib/python3.7/distutils/command/build.py", line 135, in run self.run_command(cmd_name) File "/home/he/anaconda3/lib/python3.7/distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/home/he/anaconda3/lib/python3.7/distutils/dist.py", line 985, in run_command cmd_obj.run() File "/home/he/anaconda3/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 87, in run _build_ext.run(self) File "/home/he/anaconda3/lib/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run _build_ext.build_ext.run(self) File "/home/he/anaconda3/lib/python3.7/distutils/command/build_ext.py", line 340, in run self.build_extensions() File "/home/he/anaconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 649, in build_extensions build_ext.build_extensions(self) File "/home/he/anaconda3/lib/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 194, in build_extensions self.build_extension(ext) File "/home/he/anaconda3/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 208, in build_extension _build_ext.build_extension(self, ext) File "/home/he/anaconda3/lib/python3.7/distutils/command/build_ext.py", line 534, in build_extension depends=ext.depends) File "/home/he/anaconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 478, in unix_wrap_ninja_compile with_cuda=with_cuda) File "/home/he/anaconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1233, in _write_ninja_file_and_compile_objects error_prefix='Error compiling objects for extension') File "/home/he/anaconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1529, in _run_ninja_build raise RuntimeError(message) RuntimeError: Error compiling objects for extension Could you please help me

HardworkingLittlequ commented 3 years ago

man,I have same problem in ubuntu

Have you solved your problem? I have the same problem!

2679622694 commented 3 years ago

i have the same question

caicai-creat commented 3 years ago

I have the similar problem. running build running build_ext /home/hxy/.local/lib/python3.6/site-packages/torch/utils/cpp_extension.py:335: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend. warnings.warn(msg.format('we could not find ninja.')) running develop running egg_info writing semantic_segmentation.egg-info/PKG-INFO writing dependency_links to semantic_segmentation.egg-info/dependency_links.txt writing top-level names to semantic_segmentation.egg-info/top_level.txt reading manifest file 'semantic_segmentation.egg-info/SOURCES.txt' writing manifest file 'semantic_segmentation.egg-info/SOURCES.txt' running build_ext copying build/lib.linux-x86_64-3.6/_C.cpython-36m-x86_64-linux-gnu.so -> Creating /usr/local/lib/python3.6/dist-packages/semantic-segmentation.egg-link (link to .) semantic-segmentation 0.1 is already the active version in easy-install.pth

Installed /home/hxy/awesome-semantic-segmentation-pytorch/core/nn Processing dependencies for semantic-segmentation==0.1 Finished processing dependencies for semantic-segmentation==0.1 Have you solved your problem?

MaKaNu commented 3 years ago

Same here. I am not far into the problem, but it seems to have problems to compile the extensions.

EDIT: So now I figured out my problem (seems like finishing the building process complete, with just a few deprecation warnings), but your problem might differ. I try to respond as most as I can.

@keyboardmilk: At least you printed the complete error message. Your problem seems to be your cuda version. Your Error message says: nvcc fatal : Value 'c++14' is not defined for option 'std'. This might be the problem of this issue but I am not completly shure: https://github.com/pytorch/pytorch/issues/32135

@Minghy: It seems you dont have my problem, but you also cut your Error message after the compilations try. If you have the same error message as @keyboardmilk you might have a similar problem.

My actual problem: I was missing Python.h file. It was installed for older versions of Python, but for the one I used. So my fix was simply this: sudo apt-get install Python3.8-dev

zhyunxiang commented 3 years ago

I also have the same question,have you solved? `The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "setup.py", line 49, in setup( File "E:\python\Anaconda3\envs\pytorch\lib\site-packages\setuptools__init__.py", line 153, in setup return distutils.core.setup(**attrs) File "E:\python\Anaconda3\envs\pytorch\lib\distutils\core.py", line 148, in setup dist.run_commands() File "E:\python\Anaconda3\envs\pytorch\lib\distutils\dist.py", line 966, in run_commands self.run_command(cmd) File "E:\python\Anaconda3\envs\pytorch\lib\distutils\dist.py", line 985, in run_command cmd_obj.run() File "E:\python\Anaconda3\envs\pytorch\lib\distutils\command\build.py", line 135, in run self.run_command(cmd_name) File "E:\python\Anaconda3\envs\pytorch\lib\distutils\cmd.py", line 313, in run_command self.distribution.run_command(command) File "E:\python\Anaconda3\envs\pytorch\lib\distutils\dist.py", line 985, in run_command cmd_obj.run() File "E:\python\Anaconda3\envs\pytorch\lib\site-packages\setuptools\command\build_ext.py", line 79, in run _build_ext.run(self) File "E:\python\Anaconda3\envs\pytorch\lib\distutils\command\build_ext.py", line 340, in run self.build_extensions() File "E:\python\Anaconda3\envs\pytorch\lib\site-packages\torch\utils\cpp_extension.py", line 653, in build_extensions build_ext.build_extensions(self) File "E:\python\Anaconda3\envs\pytorch\lib\distutils\command\build_ext.py", line 449, in build_extensions self._build_extensions_serial() File "E:\python\Anaconda3\envs\pytorch\lib\distutils\command\build_ext.py", line 474, in _build_extensions_serial self.build_extension(ext) File "E:\python\Anaconda3\envs\pytorch\lib\site-packages\setuptools\command\build_ext.py", line 196, in build_extension _build_ext.build_extension(self, ext) File "E:\python\Anaconda3\envs\pytorch\lib\distutils\command\build_ext.py", line 528, in build_extension objects = self.compiler.compile(sources, File "E:\python\Anaconda3\envs\pytorch\lib\site-packages\torch\utils\cpp_extension.py", line 626, in win_wrap_ninja_compile _write_ninja_file_and_compile_objects( File "E:\python\Anaconda3\envs\pytorch\lib\site-packages\torch\utils\cpp_extension.py", line 1233, in _write_ninja_file_and_compile_objects _run_ninja_build( File "E:\python\Anaconda3\envs\pytorch\lib\site-packages\torch\utils\cpp_extension.py", line 1538, in _run_ninja_build raise RuntimeError(message) from e RuntimeError: Error compiling objects for extension`

MaKaNu commented 3 years ago

@zhyunxiang As I mentioned in my solution the problem was not inside the python setup script. It was a problem in compile time of the C++ libraries.

As your message prints out : The above exception was the direct cause of the following exception: you have a similar problem inside the C++ compile time. Look up your output above this line. Somewher you might find the information you need. If the Output says: missing Python.h you probably need to install the devkit for your Python Env. Since I am using Python3.8 I needed to install python3.8-dev. Before you start installing it you might want to locate it first:

> locate Python.h
/usr/include/python3.8/Python.h

The output above is what I see on my System. If you locate it not in your venv or /usr/include/ maybe something went wrong with your pathing.

If you got a similar message as @keyboardmilk you might endup installing the wrong cuda setup.

zhyunxiang commented 3 years ago

@MaKaNu Thanks a lot! You're right, I've solved the problem.