search

zhanghang1989 / PyTorch-Encoding

A CV toolkit for my papers.
https://hangzhang.org/PyTorch-Encoding/
MIT License
2.04k stars 450 forks source link

ninja: build stopped: subcommand failed. #140

Closed sanersbug closed 5 years ago

sanersbug commented 5 years ago

After i installed ninja , When i run python main.py --dataset cifar10 --model encnetdrop --widen 8 --ncodes 32 --resume model/encnet_cifar.pth.tar --eval

it shows the error:

/home/anaconda3/lib/python3.6/site-packages/torch/utils/cpp_extension.py:118: UserWarning: 

                           !! WARNING !!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Your compiler (c++) may be ABI-incompatible with PyTorch! Please use a compiler that is ABI-compatible with GCC 4.9 and above. See https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html.

See https://gist.github.com/goldsborough/d466f43e8ffc948ff92de7486c5216d6 for instructions on how to install GCC 4.9 or higher. !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

                          !! WARNING !!

warnings.warn(ABI_INCOMPATIBILITY_WARNING.format(compiler)) Traceback (most recent call last): File "/home/anaconda3/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 759, in _build_extension_module ['ninja', '-v'], stderr=subprocess.STDOUT, cwd=build_directory) File "/home/anaconda3/lib/python3.6/subprocess.py", line 336, in check_output **kwargs).stdout File "/home/anaconda3/lib/python3.6/subprocess.py", line 418, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "main.py", line 24, in from encoding.utils import File "/home/anaconda3/lib/python3.6/site-packages/encoding/init.py", line 13, in from . import nn, functions, dilated, parallel, utils, models, datasets File "/home/anaconda3/lib/python3.6/site-packages/encoding/nn/init.py", line 12, in from .encoding import File "/home/anaconda3/lib/python3.6/site-packages/encoding/nn/encoding.py", line 18, in from ..functions import scaled_l2, aggregate, pairwise_cosine File "/home/anaconda3/lib/python3.6/site-packages/encoding/functions/init.py", line 2, in from .encoding import * File "/home/anaconda3/lib/python3.6/site-packages/encoding/functions/encoding.py", line 14, in from .. import lib File "/home/anaconda3/lib/python3.6/site-packages/encoding/lib/init.py", line 15, in ], build_directory=cpu_path, verbose=False) File "/home/anaconda3/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 514, in load with_cuda=with_cuda) File "/home/anaconda3/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 682, in _jit_compile _build_extension_module(name, build_directory) File "/home/anaconda3/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 765, in _build_extension_module name, error.output.decode())) RuntimeError: Error building extension 'enclib_cpu': [1/4] c++ -MMD -MF syncbn_cpu.o.d -DTORCH_EXTENSION_NAME=enclib_cpu -I/home/anaconda3/lib/python3.6/site-packages/torch/lib/include -I/home/anaconda3/lib/python3.6/site-packages/torch/lib/include/TH -I/home/anaconda3/lib/python3.6/site-packages/torch/lib/include/THC -I/home/anaconda3/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++11 -c /home/anaconda3/lib/python3.6/site-packages/encoding/lib/cpu/syncbn_cpu.cpp -o syncbn_cpu.o FAILED: syncbn_cpu.o c++ -MMD -MF syncbn_cpu.o.d -DTORCH_EXTENSION_NAME=enclib_cpu -I/home/anaconda3/lib/python3.6/site-packages/torch/lib/include -I/home/anaconda3/lib/python3.6/site-packages/torch/lib/include/TH -I/home/anaconda3/lib/python3.6/site-packages/torch/lib/include/THC -I/home/anaconda3/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++11 -c /home/anaconda3/lib/python3.6/site-packages/encoding/lib/cpu/syncbn_cpu.cpp -o syncbn_cpu.o /home/anaconda3/lib/python3.6/site-packages/encoding/lib/cpu/syncbn_cpu.cpp:1:26: fatal error: torch/tensor.h: No such file or directory compilation terminated. [2/4] c++ -MMD -MF roi_align_cpu.o.d -DTORCH_EXTENSION_NAME=enclib_cpu -I/home/anaconda3/lib/python3.6/site-packages/torch/lib/include -I/home/anaconda3/lib/python3.6/site-packages/torch/lib/include/TH -I/home/anaconda3/lib/python3.6/site-packages/torch/lib/include/THC -I/home/anaconda3/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++11 -c /home/anaconda3/lib/python3.6/site-packages/encoding/lib/cpu/roi_align_cpu.cpp -o roi_align_cpu.o FAILED: roi_align_cpu.o c++ -MMD -MF roi_align_cpu.o.d -DTORCH_EXTENSION_NAME=enclib_cpu -I/home/anaconda3/lib/python3.6/site-packages/torch/lib/include -I/home/anaconda3/lib/python3.6/site-packages/torch/lib/include/TH -I/home/anaconda3/lib/python3.6/site-packages/torch/lib/include/THC -I/home/anaconda3/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++11 -c /home/anaconda3/lib/python3.6/site-packages/encoding/lib/cpu/roi_align_cpu.cpp -o roi_align_cpu.o /home/anaconda3/lib/python3.6/site-packages/encoding/lib/cpu/roi_align_cpu.cpp:1:26: fatal error: torch/tensor.h: No such file or directory compilation terminated. [3/4] c++ -MMD -MF nms_cpu.o.d -DTORCH_EXTENSION_NAME=enclib_cpu -I/home/anaconda3/lib/python3.6/site-packages/torch/lib/include -I/home/anaconda3/lib/python3.6/site-packages/torch/lib/include/TH -I/home/anaconda3/lib/python3.6/site-packages/torch/lib/include/THC -I/home/anaconda3/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++11 -c /home/anaconda3/lib/python3.6/site-packages/encoding/lib/cpu/nms_cpu.cpp -o nms_cpu.o FAILED: nms_cpu.o c++ -MMD -MF nms_cpu.o.d -DTORCH_EXTENSION_NAME=enclib_cpu -I/home/anaconda3/lib/python3.6/site-packages/torch/lib/include -I/home/anaconda3/lib/python3.6/site-packages/torch/lib/include/TH -I/home/anaconda3/lib/python3.6/site-packages/torch/lib/include/THC -I/home/anaconda3/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++11 -c /home/anaconda3/lib/python3.6/site-packages/encoding/lib/cpu/nms_cpu.cpp -o nms_cpu.o /home/anaconda3/lib/python3.6/site-packages/encoding/lib/cpu/nms_cpu.cpp:1:26: fatal error: torch/tensor.h: No such file or directory compilation terminated. ninja: build stopped: subcommand failed.

zhanghang1989 commented 5 years ago

That looks like PyTorch problem.

dongzhang89 commented 5 years ago

hi, have you solved this problem ?

SuperMousse commented 5 years ago

Wow, I have faced with the same problem. But I have an gcc version of 5.4.0, I don't know why.

VanNope commented 5 years ago

Hello! im getting the same error when i try running model/download_models.py Did anyone manage to solve this issue?

zhanghang1989 commented 5 years ago

Please change torch/torch.h to torch/extension.h. This is because rapid updates in PyTorch.

VanNope commented 5 years ago

Thanks for the quick reply. Unfortunately the error message persists even after changing the name. (I changed pipPath/torch/lib/include/torch/torch.h to extension.h, where pipPath = folder that includes all pip installs).

Heres the error message i get when running model/download_models.py. I will highlight the errors in bold with gcc version5.4.0:

` /home/ev/.local/lib/python3.5/site-packages/torch/utils/cpp_extension.py:118: UserWarning: 

                           !! WARNING !!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Your compiler (c++) may be ABI-incompatible with PyTorch! Please use a compiler that is ABI-compatible with GCC 4.9 and above. See https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html.

See https://gist.github.com/goldsborough/d466f43e8ffc948ff92de7486c5216d6 for instructions on how to install GCC 4.9 or higher. !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

                          !! WARNING !!

warnings.warn(ABI_INCOMPATIBILITY_WARNING.format(compiler)) Traceback (most recent call last): File "/home/ev/.local/lib/python3.5/site-packages/torch/utils/cpp_extension.py", line 759, in _build_extension_module ['ninja', '-v'], stderr=subprocess.STDOUT, cwd=build_directory) File "/usr/lib/python3.5/subprocess.py", line 626, in check_output kwargs).stdout File "/usr/lib/python3.5/subprocess.py", line 708, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1**

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "model/download_models.py", line 1, in import encoding File "/home/ev/.local/lib/python3.5/site-packages/encoding/init.py", line 13, in from . import nn, functions, dilated, parallel, utils, models, datasets File "/home/ev/.local/lib/python3.5/site-packages/encoding/nn/init.py", line 12, in from .encoding import File "/home/ev/.local/lib/python3.5/site-packages/encoding/nn/encoding.py", line 18, in from ..functions import scaled_l2, aggregate, pairwise_cosine File "/home/ev/.local/lib/python3.5/site-packages/encoding/functions/init.py", line 2, in from .encoding import File "/home/ev/.local/lib/python3.5/site-packages/encoding/functions/encoding.py", line 14, in from .. import lib File "/home/ev/.local/lib/python3.5/site-packages/encoding/lib/init.py", line 15, in ], build_directory=cpu_path, verbose=False) File "/home/evangello/.local/lib/python3.5/site-packages/torch/utils/cpp_extension.py", line 514, in load with_cuda=with_cuda) File "/home/evangello/.local/lib/python3.5/site-packages/torch/utils/cpp_extension.py", line 682, in _jit_compile _build_extension_module(name, build_directory) File "/home/ev/.local/lib/python3.5/site-packages/torch/utils/cpp_extension.py", line 765, in _build_extension_module name, error.output.decode())) RuntimeError: Error building extension 'enclib_cpu': [1/4] c++ -MMD -MF syncbn_cpu.o.d -DTORCH_EXTENSION_NAME=enclib_cpu -I/home/ev/.local/lib/python3.5/site-packages/torch/lib/include -I/home/ev/.local/lib/python3.5/site-packages/torch/lib/include/TH -I/home/ev/.local/lib/python3.5/site-packages/torch/lib/include/THC -I/usr/include/python3.5m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++11 -c /home/ev/.local/lib/python3.5/site-packages/encoding/lib/cpu/syncbn_cpu.cpp -o syncbn_cpu.o FAILED: syncbn_cpu.o c++ -MMD -MF syncbn_cpu.o.d -DTORCH_EXTENSION_NAME=enclib_cpu -I/home/ev/.local/lib/python3.5/site-packages/torch/lib/include -I/home/ev/.local/lib/python3.5/site-packages/torch/lib/include/TH -I/home/ev/.local/lib/python3.5/site-packages/torch/lib/include/THC -I/usr/include/python3.5m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++11 -c /home/ev/.local/lib/python3.5/site-packages/encoding/lib/cpu/syncbn_cpu.cpp -o syncbn_cpu.o /home/ev/.local/lib/python3.5/site-packages/encoding/lib/cpu/syncbn_cpu.cpp:1:26: fatal error: torch/tensor.h: No such file or directory compilation terminated. [2/4] c++ -MMD -MF roi_align_cpu.o.d -DTORCH_EXTENSION_NAME=enclib_cpu -I/home/ev/.local/lib/python3.5/site-packages/torch/lib/include -I/home/ev/.local/lib/python3.5/site-packages/torch/lib/include/TH -I/home/ev/.local/lib/python3.5/site-packages/torch/lib/include/THC -I/usr/include/python3.5m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++11 -c /home/ev/.local/lib/python3.5/site-packages/encoding/lib/cpu/roi_align_cpu.cpp -o roi_align_cpu.o FAILED: roi_align_cpu.o c++ -MMD -MF roi_align_cpu.o.d -DTORCH_EXTENSION_NAME=enclib_cpu -I/home/ev/.local/lib/python3.5/site-packages/torch/lib/include -I/home/ev/.local/lib/python3.5/site-packages/torch/lib/include/TH -I/home/ev/.local/lib/python3.5/site-packages/torch/lib/include/THC -I/usr/include/python3.5m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++11 -c /home/ev/.local/lib/python3.5/site-packages/encoding/lib/cpu/roi_align_cpu.cpp -o roi_align_cpu.o /home/ev/.local/lib/python3.5/site-packages/encoding/lib/cpu/roi_align_cpu.cpp:1:26: fatal error: torch/tensor.h: No such file or directory compilation terminated. [3/4] c++ -MMD -MF nms_cpu.o.d -DTORCH_EXTENSION_NAME=enclib_cpu -I/home/ev/.local/lib/python3.5/site-packages/torch/lib/include -I/home/ev/.local/lib/python3.5/site-packages/torch/lib/include/TH -I/home/ev/.local/lib/python3.5/site-packages/torch/lib/include/THC -I/usr/include/python3.5m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++11 -c /home/ev/.local/lib/python3.5/site-packages/encoding/lib/cpu/nms_cpu.cpp -o nms_cpu.o FAILED: nms_cpu.o c++ -MMD -MF nms_cpu.o.d -DTORCH_EXTENSION_NAME=enclib_cpu -I/home/ev/.local/lib/python3.5/site-packages/torch/lib/include -I/home/ev/.local/lib/python3.5/site-packages/torch/lib/include/TH -I/home/ev/.local/lib/python3.5/site-packages/torch/lib/include/THC -I/usr/include/python3.5m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++11 -c /home/ev/.local/lib/python3.5/site-packages/encoding/lib/cpu/nms_cpu.cpp -o nms_cpu.o /home/ev/.local/lib/python3.5/site-packages/encoding/lib/cpu/nms_cpu.cpp:1:26: fatal error: torch/tensor.h: No such file or directory compilation terminated. ninja: build stopped: subcommand failed. `

zhanghang1989 commented 5 years ago

see my previous comment https://github.com/zhanghang1989/PyTorch-Encoding/issues/140#issuecomment-438373706

SuperMousse commented 5 years ago

I found that the problem is that the position of <torch/tensor.h> has been changed , it's in <torch/serialize/tensor.h> now. I changed the code and solved my prolem.

Hanhanhan11 commented 5 years ago

I found that the problem is that the position of <torch/tensor.h> has been changed , it's in <torch/serialize/tensor.h> now. I changed the code and solved my prolem.

Hi, which version of pytorch do you use??

SuperMousse commented 5 years ago

default

This is the version of pytorch. I use the master branch of Nov, 17. But I think may be v1.0 will help after change the code. ps:CUDA=9.0, CUDNN=7.0. I wish that these will help.

zhanghang1989 commented 5 years ago

Please change torch/torch.h to torch/extension.h. This is because rapid updates in PyTorch.

lfdeep commented 5 years ago

I found that the problem is that the position of <torch/tensor.h> has been changed , it's in <torch/serialize/tensor.h> now. I changed the code and solved my prolem.

Hello, i meet the same problems, i want to know how to change the code? than you

SuperMousse commented 5 years ago

I found that the problem is that the position of <torch/tensor.h> has been changed , it's in <torch/serialize/tensor.h> now. I changed the code and solved my prolem.

Hello, i meet the same problems, i want to know how to change the code? than you

At first I tried what @zhanghang1989 said " Please change torch/torch.h to torch/extension.h. " .But it didn't solve my problem. I found that the error was that <torch/tensor.h> couldnot be found, so I changed all the "#include <torch/tensor.h>" in cpp files to "#include <torch/serialize/tensor.h>". Maybe the position of "tensor.h" can be different, you can change the path in your own system. Hope this will help.

XiaofanLinUS commented 5 years ago

I found that the problem is that the position of <torch/tensor.h> has been changed , it's in <torch/serialize/tensor.h> now. I changed the code and solved my prolem.

Hello, i meet the same problems, i want to know how to change the code? than you

At first I tried what @zhanghang1989 said " Please change torch/torch.h to torch/extension.h. " .But it didn't solve my problem. I found that the error was that <torch/tensor.h> couldnot be found, so I changed all the "#include <torch/tensor.h>" in cpp files to "#include <torch/serialize/tensor.h>". Maybe the position of "tensor.h" can be different, you can change the path in your own system. Hope this will help.

I have the same problem as yours.

Cverlpeng commented 5 years ago

Please change torch/torch.h to torch/extension.h. This is because rapid updates in PyTorch. image

hi, Rename torch.h to extension.h ?

tangky22 commented 2 years ago

Please change torch/torch.h to torch/extension.h. This is because rapid updates in PyTorch. image

hi, Rename torch.h to extension.h ?

hi, have you solve the problem? I met the same problem in pytorch=1.7 torch-encoding=1.2.1

zhanghang1989 commented 2 years ago

Please uninstall the torch-encoding and reinstall it following the instruction at https://hangzhang.org/PyTorch-Encoding/notes/compile.html

tangky22 commented 2 years ago

Please uninstall the torch-encoding and reinstall it following the instruction at https://hangzhang.org/PyTorch-Encoding/notes/compile.html

Thanks for your reply! Should I reinstall the torch-encoding using Ubuntu? I didn't find the instruction for Windows10 using Anaconda