Compilation error of cuda/setup.py on windows

[Xieyuan0018]

Hi, I tried to build the extension using your sc. I managed to build from cpp/setup.py and was able to import it. However, I just cannot build from cuda/setup.py. I tried the solution mentioned in other posts i.e. fmax to ::fmax, casting etc, none of them worked.

The error message: E:/Miniconda/envs/pointnetpy/lib/site-packages/torch/include\torch/csrc/jit/argument_spec.h(161): error: member "torch::jit::ArgumentSpecCreator::DEPTH_LIMIT" may not be initialized 1 error detected in the compilation of "C:/Users/XIEYUA~1/AppData/Local/Temp/tmpxft_00002f00_00000000-10_lltm_cuda_kernel.cpp1.ii". error: command 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin\nvcc.exe' failed with exit status 1

• OS: Windows10 Enterprise Version1809
• PyTorch version:
• How you installed PyTorch (conda, pip, source): conda
• Python version: 1.1.0
• CUDA/cuDNN version: 10.1
• GPU models and configuration: GTX 1080Ti
• GCC version (if compiling from source): N.A.

Any thoughts or suggestions would be greatly appreciated!

p.s. FYI, I failed to compile using jit.py for both cpp and cuda version.

The error log: Using C:\Users\XIEYUA~1\AppData\Local\Temp\torch_extensions as PyTorch extensions root... Emitting ninja build file C:\Users\XIEYUA~1\AppData\Local\Temp\torch_extensions\lltm_cpp\build.ninja... Building extension module lltm_cpp... [1/2] cl /showIncludes -DTORCH_EXTENSION_NAME=lltm_cpp -DTORCH_API_INCLUDE_EXTENSION_H -IE:\Miniconda\envs\pointnetpy\lib\site-packages\torch\include -IE:\Miniconda\envs\pointnetpy\lib\site-packages\torch\include\torch\csrc\api\include -IE:\Miniconda\envs\pointnetpy\lib\site-packages\torch\include\TH -IE:\Miniconda\envs\pointnetpy\lib\site-packages\torch\include\THC -IE:\Miniconda\envs\pointnetpy\Include -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++11 -c D:\Github\extension-cpp\cpp\lltm.cpp /Folltm.o FAILED: lltm.o cl /showIncludes -DTORCH_EXTENSION_NAME=lltm_cpp -DTORCH_API_INCLUDE_EXTENSION_H -IE:\Miniconda\envs\pointnetpy\lib\site-packages\torch\include -IE:\Miniconda\envs\pointnetpy\lib\site-packages\torch\include\torch\csrc\api\include -IE:\Miniconda\envs\pointnetpy\lib\site-packages\torch\include\TH -IE:\Miniconda\envs\pointnetpy\lib\site-packages\torch\include\THC -IE:\Miniconda\envs\pointnetpy\Include -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++11 -c D:\Github\extension-cpp\cpp\lltm.cpp /Folltm.o Microsoft (R) C/C++ Optimizing Compiler Version 19.16.27027.1 for x64 Copyright (C) Microsoft Corporation. All rights reserved.

cl : Command line warning D9002 : ignoring unknown option '-fPIC' cl : Command line warning D9002 : ignoring unknown option '-std=c++11' E:\Miniconda\envs\pointnetpy\lib\site-packages\torch\include\torch\csrc\api\include\torch/cuda.h(5): fatal error C1083: Cannot open include file: 'cstddef': No such file or directory ninja: build stopped: subcommand failed. Traceback (most recent call last): File "E:\Miniconda\envs\pointnetpy\lib\site-packages\torch\utils\cpp_extension.py", line 949, in _build_extension_module check=True) File "E:\Miniconda\envs\pointnetpy\lib\subprocess.py", line 438, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "jit.py", line 2, in lltm_cpp = load(name="lltm_cpp", sources=["lltm.cpp"], verbose=True) File "E:\Miniconda\envs\pointnetpy\lib\site-packages\torch\utils\cpp_extension.py", line 644, in load is_python_module) File "E:\Miniconda\envs\pointnetpy\lib\site-packages\torch\utils\cpp_extension.py", line 813, in _jit_compile with_cuda=with_cuda) File "E:\Miniconda\envs\pointnetpy\lib\site-packages\torch\utils\cpp_extension.py", line 866, in _write_ninja_file_and_build _build_extension_module(name, build_directory, verbose) File "E:\Miniconda\envs\pointnetpy\lib\site-packages\torch\utils\cpp_extension.py", line 962, in _build_extension_module raise RuntimeError(message) RuntimeError: Error building extension 'lltm_cpp'

[huoshuaiUSTC]

I also encountered this problem. How did you solve it?

[RaphaelRoyerRivard]

I also get the error with jit and CUDA v10.0 when running /cuda/setup.py. I'd very much like to know the solution to this problem.

OS: Windows 10 PyTorch version: 1.1.0 How you installed PyTorch (conda, pip, source): conda Python version: 3.7.3 CUDA/cuDNN version: 10.0/7.6.0 GPU models and configuration: RTX 2070 GCC version (if compiling from source): 8.2.0

[wantsjean]

I've "solved" the problem when I seach the whole torch folder, it shows that there's only one DEPTH_LIMIT, so I just comment the (161) line where error raised in ...torch/include\torch/csrc/jit/argument_spec.h and when I compiled it again, it shows another error: <env_lib_dir>\site-packages\torch\lib\include\pybind11\cast.h(1393): error: expression must be a pointer to a complete object type then I followed someone's solution go straight into the cast.h file and make the following changes replace explicit operator type&() { return *(this->value); }with explicit operator type& () { *((type*)(this->value)); } and it finally worked.

[elnazsn1988]

@Xieyuan0018 @wantsjean Hi, Im encountering the same file compilation error :

1 error detected in the compilation of "C:/Users/XIEYUA~1/AppData/Local/Temp/tmpxft_00002f00_00000000-10_lltm_cuda_kernel.cpp1.ii". error: command 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin\nvcc.exe' failed with exit status 1

Except minus the first part - my error doesnt point me to the Depth variable prior to the error, instead I get this:

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin\nvcc.exe -c C:\projects\pytorch\pytorch\detectron2\detectron2\layers\csrc\deformable\deform_conv_cuda.cu -o build\temp.win-amd64-3.7\Release\projects\pytorch\pytorch\detectron2\detectron2\layers\csrc\deformable\deform_conv_cuda.obj -IC:\projects\pytorch\pytorch\detectron2\detectron2\layers\csrc -IC:\Users\aesnj\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\include -IC:\Users\aesnj\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\aesnj\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\include\TH -IC:\Users\aesnj\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\include" -IC:\Users\aesnj\AppData\Local\Programs\Python\Python37\include -IC:\Users\aesnj\AppData\Local\Programs\Python\Python37\include "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.23.28105\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.23.28105\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\cppwinrt" "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.23.28105\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.23.28105\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\cppwinrt" -Xcompiler /EHsc -Xcompiler /wd4819 -Xcompiler /MD -DCUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr -DCUDA_HAS_FP16=1 -DCUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=sm_75 deform_conv_cuda.cu

I did implement the changes mentioned above for argument_spec and cost.h files in case and tried another clone version of detectron2 : https://github.com/facebookresearch/detectron2/issues/9 , same error.

Can I ask if you guys have used Conda or straight python? I use Admin VS 2019 tools command prompt which initializes correctly, and when testing my nvcc build, it seems to be ok. Am trying Conda now but am wondering if I can isolate the issue and not depend on conda?

[jamesr66a]

This should be fixed now by https://github.com/pytorch/pytorch/pull/33002

[mdcramer]

I have Pytorch 1.4.0 installed, and have updated everything else, and I'm still getting torch/csrc/jit/argument_spec.h(190): error: member "torch::jit::ArgumentSpecCreator::DEPTH_LIMIT" may not be initialized.

I did, however, comment out line 190 in argument_spec.h and the error disappeared.

Update: I read at https://github.com/facebookresearch/detectron2/issues/9#issuecomment-549013090 that line 190 should be changed to static const size_t DEPTH_LIMIT = 128;, so I did that and it seems like this error is gone.

[jamesr66a]

@mdcramer the fix was landed after the 1.4.0 branch was cut. It will be present in either the nightly build or the 1.5.0 release, when that happens

[Anandsrivastava90]

Hi I am intalling Detectron2 in Windows10 , when I run python setup.py install I am getting this error .Please help me out

Microsoft Visual Studio 2019 pytorch 1.5 cuda 10.1

Traceback (most recent call last): File "D:\anaconda\envs\detectron_project\lib\site-packages\torch\utils\cpp_extension.py", line 1400, in _run_ninja_build check=True) File "D:\anaconda\envs\detectron_project\lib\subprocess.py", line 438, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "setup.py", line 115, in cmdclass={"build_ext": torch.utils.cpp_extension.BuildExtension}, File "D:\anaconda\envs\detectron_project\lib\site-packages\setuptools__init__.py", line 165, in setup return distutils.core.setup(**attrs) File "D:\anaconda\envs\detectron_project\lib\distutils\core.py", line 148, in setup dist.run_commands() File "D:\anaconda\envs\detectron_project\lib\distutils\dist.py", line 955, in run_commands self.run_command(cmd) File "D:\anaconda\envs\detectron_project\lib\distutils\dist.py", line 974, in run_command cmd_obj.run() File "D:\anaconda\envs\detectron_project\lib\site-packages\setuptools\command\install.py", line 67, in run self.do_egg_install() File "D:\anaconda\envs\detectron_project\lib\site-packages\setuptools\command\install.py", line 109, in do_egg_install self.run_command('bdist_egg') File "D:\anaconda\envs\detectron_project\lib\distutils\cmd.py", line 313, in run_command self.distribution.run_command(command) File "D:\anaconda\envs\detectron_project\lib\distutils\dist.py", line 974, in run_command cmd_obj.run() File "D:\anaconda\envs\detectron_project\lib\site-packages\setuptools\command\bdist_egg.py", line 174, in run cmd = self.call_command('install_lib', warn_dir=0) File "D:\anaconda\envs\detectron_project\lib\site-packages\setuptools\command\bdist_egg.py", line 160, in call_command self.run_command(cmdname) File "D:\anaconda\envs\detectron_project\lib\distutils\cmd.py", line 313, in run_command self.distribution.run_command(command) File "D:\anaconda\envs\detectron_project\lib\distutils\dist.py", line 974, in run_command cmd_obj.run() File "D:\anaconda\envs\detectron_project\lib\site-packages\setuptools\command\install_lib.py", line 11, in run self.build() File "D:\anaconda\envs\detectron_project\lib\distutils\command\install_lib.py", line 107, in build self.run_command('build_ext') File "D:\anaconda\envs\detectron_project\lib\distutils\cmd.py", line 313, in run_command self.distribution.run_command(command) File "D:\anaconda\envs\detectron_project\lib\distutils\dist.py", line 974, in run_command cmd_obj.run() File "D:\anaconda\envs\detectron_project\lib\site-packages\setuptools\command\build_ext.py", line 87, in run _build_ext.run(self) File "D:\anaconda\envs\detectron_project\lib\site-packages\Cython\Distutils\old_build_ext.py", line 186, in run _build_ext.build_ext.run(self) File "D:\anaconda\envs\detectron_project\lib\distutils\command\build_ext.py", line 339, in run self.build_extensions() File "D:\anaconda\envs\detectron_project\lib\site-packages\torch\utils\cpp_extension.py", line 580, in build_extensions build_ext.build_extensions(self) File "D:\anaconda\envs\detectron_project\lib\site-packages\Cython\Distutils\old_build_ext.py", line 195, in build_extensions _build_ext.build_ext.build_extensions(self) File "D:\anaconda\envs\detectron_project\lib\distutils\command\build_ext.py", line 448, in build_extensions self._build_extensions_serial() File "D:\anaconda\envs\detectron_project\lib\distutils\command\build_ext.py", line 473, in _build_extensions_serial self.build_extension(ext) File "D:\anaconda\envs\detectron_project\lib\site-packages\setuptools\command\build_ext.py", line 208, in build_extension _build_ext.build_extension(self, ext) File "D:\anaconda\envs\detectron_project\lib\distutils\command\build_ext.py", line 533, in build_extension depends=ext.depends) File "D:\anaconda\envs\detectron_project\lib\site-packages\torch\utils\cpp_extension.py", line 562, in win_wrap_ninja_compile with_cuda=with_cuda) File "D:\anaconda\envs\detectron_project\lib\site-packages\torch\utils\cpp_extension.py", line 1140, in _write_ninja_file_and_compile_objects error_prefix='Error compiling objects for extension') File "D:\anaconda\envs\detectron_project\lib\site-packages\torch\utils\cpp_extension.py", line 1413, in _run_ninja_build raise RuntimeError(message) RuntimeError: Error compiling objects for extension

[tehreemnaqvi]

I got the same error.

I tried a lot to fix this but couldn't debug the errors.

How to fix this error?

Please help!!

Thanks

[zimonitrome]

Hi I am intalling Detectron2 in Windows10 , when I run python setup.py install I am getting this error .Please help me out

Microsoft Visual Studio 2019 pytorch 1.5 cuda 10.1

Traceback (most recent call last): File "D:\anaconda\envs\detectron_project\lib\site-packages\torch\utils\cpp_extension.py", line 1400, in _run_ninja_build check=True) File "D:\anaconda\envs\detectron_project\lib\subprocess.py", line 438, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "setup.py", line 115, in cmdclass={"build_ext": torch.utils.cpp_extension.BuildExtension}, File "D:\anaconda\envs\detectron_project\lib\site-packages\setuptoolsinit.py", line 165, in setup return distutils.core.setup(**attrs) File "D:\anaconda\envs\detectron_project\lib\distutils\core.py", line 148, in setup dist.run_commands() File "D:\anaconda\envs\detectron_project\lib\distutils\dist.py", line 955, in run_commands self.run_command(cmd) File "D:\anaconda\envs\detectron_project\lib\distutils\dist.py", line 974, in run_command cmd_obj.run() File "D:\anaconda\envs\detectron_project\lib\site-packages\setuptools\command\install.py", line 67, in run self.do_egg_install() File "D:\anaconda\envs\detectron_project\lib\site-packages\setuptools\command\install.py", line 109, in do_egg_install self.run_command('bdist_egg') File "D:\anaconda\envs\detectron_project\lib\distutils\cmd.py", line 313, in run_command self.distribution.run_command(command) File "D:\anaconda\envs\detectron_project\lib\distutils\dist.py", line 974, in run_command cmd_obj.run() File "D:\anaconda\envs\detectron_project\lib\site-packages\setuptools\command\bdist_egg.py", line 174, in run cmd = self.call_command('install_lib', warn_dir=0) File "D:\anaconda\envs\detectron_project\lib\site-packages\setuptools\command\bdist_egg.py", line 160, in call_command self.run_command(cmdname) File "D:\anaconda\envs\detectron_project\lib\distutils\cmd.py", line 313, in run_command self.distribution.run_command(command) File "D:\anaconda\envs\detectron_project\lib\distutils\dist.py", line 974, in run_command cmd_obj.run() File "D:\anaconda\envs\detectron_project\lib\site-packages\setuptools\command\install_lib.py", line 11, in run self.build() File "D:\anaconda\envs\detectron_project\lib\distutils\command\install_lib.py", line 107, in build self.run_command('build_ext') File "D:\anaconda\envs\detectron_project\lib\distutils\cmd.py", line 313, in run_command self.distribution.run_command(command) File "D:\anaconda\envs\detectron_project\lib\distutils\dist.py", line 974, in run_command cmd_obj.run() File "D:\anaconda\envs\detectron_project\lib\site-packages\setuptools\command\build_ext.py", line 87, in run _build_ext.run(self) File "D:\anaconda\envs\detectron_project\lib\site-packages\Cython\Distutils\old_build_ext.py", line 186, in run _build_ext.build_ext.run(self) File "D:\anaconda\envs\detectron_project\lib\distutils\command\build_ext.py", line 339, in run self.build_extensions() File "D:\anaconda\envs\detectron_project\lib\site-packages\torch\utils\cpp_extension.py", line 580, in build_extensions build_ext.build_extensions(self) File "D:\anaconda\envs\detectron_project\lib\site-packages\Cython\Distutils\old_build_ext.py", line 195, in build_extensions _build_ext.build_ext.build_extensions(self) File "D:\anaconda\envs\detectron_project\lib\distutils\command\build_ext.py", line 448, in build_extensions self._build_extensions_serial() File "D:\anaconda\envs\detectron_project\lib\distutils\command\build_ext.py", line 473, in _build_extensions_serial self.build_extension(ext) File "D:\anaconda\envs\detectron_project\lib\site-packages\setuptools\command\build_ext.py", line 208, in build_extension _build_ext.build_extension(self, ext) File "D:\anaconda\envs\detectron_project\lib\distutils\command\build_ext.py", line 533, in build_extension depends=ext.depends) File "D:\anaconda\envs\detectron_project\lib\site-packages\torch\utils\cpp_extension.py", line 562, in win_wrap_ninja_compile with_cuda=with_cuda) File "D:\anaconda\envs\detectron_project\lib\site-packages\torch\utils\cpp_extension.py", line 1140, in _write_ninja_file_and_compile_objects error_prefix='Error compiling objects for extension') File "D:\anaconda\envs\detectron_project\lib\site-packages\torch\utils\cpp_extension.py", line 1413, in _run_ninja_build raise RuntimeError(message) RuntimeError: Error compiling objects for extension

Still having this same issue. No down-grading seems to help.

[zimonitrome]

If anyone else has the same problem, I wrote about it on https://discuss.pytorch.org/t/can-seemingly-not-build-cpp-cuda-extensions-for-any-version-of-pytorch-cudatoolkit/135073/2

This was actually an error with Visual Stuio being installed incorrectly. I uninstalled all Vsiual Studio components, removed the environtment variables (named ones and from Path), and removed all files on my disk. Then I installed VS 2019 (As that is the latest supported version by PyTorch as of making this post) and installed the C++ dev tools (SDK, build tools) with it. I then had to re-install Pytorch in conda (conda uninstall pytorch torchvision torchaudio cudatoolkit) and then installed them again, simply replacing the existing packages did not work.

After all this, it finally worked.