search

HRNet / HRNet-Semantic-Segmentation

The OCR approach is rephrased as Segmentation Transformer: https://arxiv.org/abs/1909.11065. This is an official implementation of semantic segmentation for HRNet. https://arxiv.org/abs/1908.07919
Other
3.12k stars 685 forks source link

RuntimeError: Ninja is required to load C++ extensions #20

Closed GuangyanZhang closed 4 years ago

GuangyanZhang commented 5 years ago

您好，首先我出现这样的问题： RuntimeError: Ninja is required to load C++ extensions 然后我pip install ninja成功以后 又出现这样的问题： /usr/local/lib/python3.5/dist-packages/torch/utils/cpp_extension.py:118: UserWarning:

                           !! WARNING !!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Your compiler (c++) may be ABI-incompatible with PyTorch! Please use a compiler that is ABI-compatible with GCC 4.9 and above. See https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html.

See https://gist.github.com/goldsborough/d466f43e8ffc948ff92de7486c5216d6 for instructions on how to install GCC 4.9 or higher. !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

                          !! WARNING !!

warnings.warn(ABI_INCOMPATIBILITY_WARNING.format(compiler)) Traceback (most recent call last): File "tools/train.py", line 27, in import models File "/data/HRNet-Semantic-Segmentation-master/tools/../lib/models/init.py", line 11, in import models.seg_hrnet File "/data/HRNet-Semantic-Segmentation-master/tools/../lib/models/seg_hrnet.py", line 22, in from .sync_bn.inplace_abn.bn import InPlaceABNSync File "/data/HRNet-Semantic-Segmentation-master/tools/../lib/models/sync_bn/init.py", line 1, in from .inplace_abn import bn File "/data/HRNet-Semantic-Segmentation-master/tools/../lib/models/sync_bn/inplace_abn/init.py", line 1, in from .bn import ABN, InPlaceABN, InPlaceABNSync File "/data/HRNet-Semantic-Segmentation-master/tools/../lib/models/sync_bn/inplace_abn/bn.py", line 14, in from functions import * File "/data/HRNet-Semantic-Segmentation-master/lib/models/sync_bn/inplace_abn/functions.py", line 16, in extra_cuda_cflags=["--expt-extended-lambda"]) File "/usr/local/lib/python3.5/dist-packages/torch/utils/cpp_extension.py", line 514, in load with_cuda=with_cuda) File "/usr/local/lib/python3.5/dist-packages/torch/utils/cpp_extension.py", line 690, in _jit_compile return _import_module_from_library(name, build_directory) File "/usr/local/lib/python3.5/dist-packages/torch/utils/cpp_extension.py", line 773, in _import_module_from_library return imp.load_module(module_name, file, path, description) File "/usr/lib/python3.5/imp.py", line 242, in load_module return load_dynamic(name, filename, file) File "/usr/lib/python3.5/imp.py", line 342, in load_dynamic return _load(spec) ImportError: /tmp/torch_extensions/inplace_abn/inplace_abn.so: undefined symbol: _ZN2at5ErrorC1ENS_14SourceLocationESs

请问这个BN和pytorch是要同步编译吗？我的pytorch==0.4.1

GuangyanZhang commented 5 years ago

我该怎么操作才能解决这个问题呢？希望您能指导一下，谢谢

sunke123 commented 5 years ago

You can try this docker: kesun/pose:pytorch-0.4.1-bn-cudnn-off If possible, you adopt the official sync-bn provided by pytorch.

GuangyanZhang commented 5 years ago

这个docker "kesun/pose:pytorch-0.4.1-bn-cudnn-off" 可以提供一个链接吗？谢谢！

sunke123 commented 5 years ago

https://cloud.docker.com/u/kesun/repository/docker/kesun/pose

I think you can pull this docker without any link.

GuangyanZhang commented 5 years ago

好的，非常感谢

sde123 commented 5 years ago

hi your website 'https://cloud.docker.com/u/kesun/repository/docker/kesun/pose' is not working now. Could you tell me what is wrong?

GuangyanZhang commented 5 years ago

This site maybe requires over the wall,I think

sde123 commented 5 years ago

Hi I can visit google,but I cannot visit this website,Therefore, it is not the reason of wall

GuangyanZhang commented 5 years ago

I think website is limitted,you can wait a moment.......

sunke123 commented 5 years ago

@GuangyanZhang @sde123 Sorry about that. https://hub.docker.com/r/kesun/pose/.

However, I have no idea why you need to visit this website. You can just run "docker pull kesun/pose:pytorch-0.4.1-bn-cudnn-off" to pull this image.

sde123 commented 5 years ago

Ok，Do you solve the above problem of BN and pytorch that you said？

GuangyanZhang commented 5 years ago

my website is limitted,so I want download by other computer.but I find that my environment cann't use docker.....

GuangyanZhang commented 5 years ago

@sde123 I don't,I should adopt the official sync-bn provided by pytorch.that's bad.

GuangyanZhang commented 5 years ago

@sde123 你好，之前那个搞定了，现在又出现这样的问题，我是不是哪里路径不对或者参数不对呢？，能指导下吗？ File "tools/train.py", line 245, in main() File "tools/train.py", line 56, in main args = parse_args() File "tools/train.py", line 51, in parse_args update_config(config, args) File "/data/HRNet-Semantic-Segmentation-master/tools/../lib/config/default.py", line 114, in update_config cfg.merge_from_file(args.cfg) File "/usr/local/lib/python3.5/dist-packages/yacs/config.py", line 171, in merge_from_file self.merge_from_other_cfg(cfg) File "/usr/local/lib/python3.5/dist-packages/yacs/config.py", line 175, in merge_from_other_cfg _merge_a_into_b(cfg_other, self, self, []) File "/usr/local/lib/python3.5/dist-packages/yacs/config.py", line 401, in _merge_a_into_b _merge_a_into_b(v, b[k], root, key_list + [k]) File "/usr/local/lib/python3.5/dist-packages/yacs/config.py", line 401, in _merge_a_into_b _merge_a_into_b(v, b[k], root, key_list + [k]) File "/usr/local/lib/python3.5/dist-packages/yacs/config.py", line 392, in _merge_a_into_b raise KeyError("Non-existent config key: {}".format(full_key)) KeyError: 'Non-existent config key: MODEL.EXTRA.STAGE4'

jwnirvana commented 5 years ago

I am confused by this problem(ninjia),too. Could you teach me to solve it,please?

GuangyanZhang commented 5 years ago

you should install pytorch 0.4.0 and ninja~

frostinassiky commented 5 years ago

@GuangyanZhang Did you check your config (yaml) file? Does it contain STAGE4?

yanli-star commented 5 years ago

@GuangyanZhang 你好，我安装pytorch0.4.1和您出现了同样的问题，又安装了pytorch0.4.0和ninja，出现了下面这样的问题，请问您出现过吗，怎么解决呢？ File "tools/train.py", line 27, in import models File "/root/userfolder/code/HRNet-Seg/tools/../lib/models/init.py", line 11, in import models.seg_hrnet File "/root/userfolder/code/HRNet-Seg/tools/../lib/models/seg_hrnet.py", line 22, in from .sync_bn.inplace_abn.bn import InPlaceABNSync File "/root/userfolder/code/HRNet-Seg/tools/../lib/models/sync_bn/init.py", line 1, in from .inplace_abn import bn File "/root/userfolder/code/HRNet-Seg/tools/../lib/models/sync_bn/inplace_abn/init.py", line 1, in from .bn import ABN, InPlaceABN, InPlaceABNSync File "/root/userfolder/code/HRNet-Seg/tools/../lib/models/sync_bn/inplace_abn/bn.py", line 14, in from functions import * File "/root/userfolder/code/HRNet-Seg/tools/../lib/models/sync_bn/inplace_abn/functions.py", line 16, in extra_cuda_cflags=["--expt-extended-lambda"]) File "/root/userfolder/anaconda3/envs/python27/lib/python2.7/site-packages/torch/utils/cpp_extension.py", line 501, in load _build_extension_module(name, build_directory) File "/root/userfolder/anaconda3/envs/python27/lib/python2.7/site-packages/torch/utils/cpp_extension.py", line 582, in _build_extension_module name, error.output.decode())) RuntimeError: Error building extension 'inplace_abn': ninja: warning: bad deps log signature or version; starting over [1/4] /usr/local/cuda:/bin/nvcc -DTORCH_EXTENSION_NAME=inplace_abn -I/root/userfolder/anaconda3/envs/python27/lib/python2.7/site-packages/torch/lib/include -I/root/userfolder/anaconda3/envs/python27/lib/python2.7/site-packages/torch/lib/include/TH -I/root/userfolder/anaconda3/envs/python27/lib/python2.7/site-packages/torch/lib/include/THC -I/usr/local/cuda:/include -I/root/userfolder/anaconda3/envs/python27/include/python2.7 --compiler-options '-fPIC' --expt-extended-lambda -std=c++11 -c /root/userfolder/code/HRNet-Seg/lib/models/sync_bn/inplace_abn/src/inplace_abn_cuda.cu -o inplace_abn_cuda.cuda.o FAILED: inplace_abn_cuda.cuda.o /usr/local/cuda:/bin/nvcc -DTORCH_EXTENSION_NAME=inplace_abn -I/root/userfolder/anaconda3/envs/python27/lib/python2.7/site-packages/torch/lib/include -I/root/userfolder/anaconda3/envs/python27/lib/python2.7/site-packages/torch/lib/include/TH -I/root/userfolder/anaconda3/envs/python27/lib/python2.7/site-packages/torch/lib/include/THC -I/usr/local/cuda:/include -I/root/userfolder/anaconda3/envs/python27/include/python2.7 --compiler-options '-fPIC' --expt-extended-lambda -std=c++11 -c /root/userfolder/code/HRNet-Seg/lib/models/sync_bn/inplace_abn/src/inplace_abn_cuda.cu -o inplace_abn_cuda.cuda.o /bin/sh: 1: /usr/local/cuda:/bin/nvcc: not found [2/4] c++ -MMD -MF inplace_abn_cpu.o.d -DTORCH_EXTENSION_NAME=inplace_abn -I/root/userfolder/anaconda3/envs/python27/lib/python2.7/site-packages/torch/lib/include -I/root/userfolder/anaconda3/envs/python27/lib/python2.7/site-packages/torch/lib/include/TH -I/root/userfolder/anaconda3/envs/python27/lib/python2.7/site-packages/torch/lib/include/THC -I/usr/local/cuda:/include -I/root/userfolder/anaconda3/envs/python27/include/python2.7 -fPIC -std=c++11 -O3 -c /root/userfolder/code/HRNet-Seg/lib/models/sync_bn/inplace_abn/src/inplace_abn_cpu.cpp -o inplace_abn_cpu.o [3/4] c++ -MMD -MF inplace_abn.o.d -DTORCH_EXTENSION_NAME=inplace_abn -I/root/userfolder/anaconda3/envs/python27/lib/python2.7/site-packages/torch/lib/include -I/root/userfolder/anaconda3/envs/python27/lib/python2.7/site-packages/torch/lib/include/TH -I/root/userfolder/anaconda3/envs/python27/lib/python2.7/site-packages/torch/lib/include/THC -I/usr/local/cuda:/include -I/root/userfolder/anaconda3/envs/python27/include/python2.7 -fPIC -std=c++11 -O3 -c /root/userfolder/code/HRNet-Seg/lib/models/sync_bn/inplace_abn/src/inplace_abn.cpp -o inplace_abn.o ninja: build stopped: subcommand failed.

GuangyanZhang commented 5 years ago

@Frostinassiky you should change the default.py~

GuangyanZhang commented 5 years ago

@YanYunYun ,没有出现过，这个应该还是ninja的问题

sxhsunday commented 4 years ago

@YanYunYun
Seg/lib/models/sync_bn/inplace_abn/src/inplace_abn.cpp -o inplace_abn.o ninja: build stopped: subcommand failed. did you solve this problem?

yanli-star commented 4 years ago

@sxhsunday
I solved this issue by installing pytorch 1.1 and using pytorch-v1.1 branch.

sxhsunday commented 4 years ago

@YanYunYun Thank you very much. it is work now.

kaishijeng commented 4 years ago

@sxhsunday @YanYunYun

I install pytorch 1.1 using conda and checkout pytorch v1.1 branch. However, I still got the same error. Can you provide instruction how you install pytorch 1.1 and checkout pytorch v1.1 branch? ALso which version ninja fo you use?

Thanks,

neverstoplearn commented 4 years ago

@sxhsunday @YanYunYun

I install pytorch 1.1 using conda and checkout pytorch v1.1 branch. However, I still got the same error. Can you provide instruction how you install pytorch 1.1 and checkout pytorch v1.1 branch? ALso which version ninja fo you use?

Thanks,

I have the same error as you have you solve it? thanks,

kaizen0890 commented 4 years ago

I found that this error relates to version matching problem, specifying between Torch and ninja version. Therefore I solved this problem by finding ninja version which matching with Torch version. My environment as show as below: Ubuntu: 16.04 Gcc: 6.5.0 Python: 3.5.2 Torch: 0.4.1 ninja: 1.8.2 Hopefully that it can help you guys!

iris0329 commented 4 years ago

@sxhsunday I solved this issue by installing pytorch 1.1 and using pytorch-v1.1 branch.

It works!