search

HRNet / HRNet-Semantic-Segmentation

The OCR approach is rephrased as Segmentation Transformer: https://arxiv.org/abs/1909.11065. This is an official implementation of semantic segmentation for HRNet. https://arxiv.org/abs/1908.07919
Other
3.12k stars 685 forks source link

ImportError: No module named 'inplace_abn' #141

Closed InternetMaster1 closed 4 years ago

InternetMaster1 commented 4 years ago

I tried Image Segmentation using the " HRNetV2-W18-Small-v2 " small model with cityscape dataset.

I haveinstall all modules mentioned in requirement.txt file with the matching version of the modules. My config is as follows - python 3.6 cuda 9.2 ninja 1.8.2 pytorch 0.4.1

I had done the steps till data preparations and then I tried to train using following command,

python tools/train.py --cfg experiments/cityscapes/seg_hrnet_w18_small_v1_512x1024_sgd_lr1e-2_wd5e-4_bs_12_epoch484

I am getting the below error.

warnings.warn(ABI_INCOMPATIBILITY_WARNING.format(compiler)) Traceback (most recent call last): 
File "/usr/local/lib/python3.6/dist-packages/torch/utils/cpp_extension.py", line 759, in _build_extension_module ['ninja', '-v'], stderr=subprocess.STDOUT, cwd=build_directory) 
File "/usr/lib/python3.6/subprocess.py", line 356, in check_output **kwargs).stdout 
File "/usr/lib/python3.6/subprocess.py", line 438, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/torch/utils/cpp_extension.py", line 514, in load with_cuda=with_cuda) 
File "/usr/local/lib/python3.6/dist-packages/torch/utils/cpp_extension.py", line 690, in _jit_compile return _import_module_from_library(name, build_directory) 
File "/usr/local/lib/python3.6/dist-packages/torch/utils/cpp_extension.py", line 770, in _import_module_from_library file, path, description = imp.find_module(module_name, [path]) 
File "/usr/lib/python3.6/imp.py", line 297, in find_module raise ImportError(_ERR_MSG.format(name), name=name)
ImportError: No module named 'inplace_abn'

To solve this, I have tried with diff version that match with ninja and cuda, but no luck. Any help please!

InternetMaster1 commented 4 years ago

Any help please...

@pgrimaud @leoxiaobin @sunke123 @bearcatt

sunke123 commented 4 years ago

The HRNet small model is trained based on Pytorch-v1.1 and official sync-bn is used (inplace-abn is not used here).

You can try our pytorch-v1.1 branch.

InternetMaster1 commented 4 years ago

@sunke123 Many thanks, I will give it a try!

Just a couple more questions :

1) Is there "HRNet + OCR + SegFix" option available for HRNetV2-W18-Small-v2?

The addition of OCR & Segfix would improve the accuracy of "HRNetV2-W18-Small-v2" further?

2) Is "HRNetV2-W18-Small-v2" SOTA for lightweight segmentation?

Comparison of mIOU on Cityscapes.

Why is U-HarDNet-70 at the top of paperswithcode SOTA list for real-time segmentation?

I am looking to implement high-quality semantic segmentation on a mobile device for accurate human segmentation for still images (i.e. non-realtime). Would "HRNetV2-W18-Small-v2" be a great option for this?

InternetMaster1 commented 4 years ago

@sunke123

I tried pytorch-v1.1 branch and installed the following exact configuration :+1:

PyTorch=1.1.0 EasyDict==1.7 opencv-python==3.4.1.15 shapely==1.6.4 Cython scipy pandas pyyaml json_tricks scikit-image yacs>=0.1.5 tensorboardX>=1.6 tqdm ninja given in following link https://github.com/HRNet/HRNet-Semantic-Segmentation/blob/pytorch-v1.1/requirements.txt

I am still getting same error "No module named 'inplace_abn'" while training

You said

The HRNet small model is trained based on Pytorch-v1.1 and official sync-bn is used (inplace-abn is not used here)

but in this file, I noticed that "inplace_abn" is imported

from .inplace_abn import bn

Do I need to do any changes here?

sunke123 commented 4 years ago

@InternetMaster1 You can remove the sync-bn folder directly.

  1. Is there "HRNet + OCR + SegFix" option available for HRNetV2-W18-Small-v2? I have not tried this, but I think is is OK. @hsfzxjy Could you try to implement OCR module on the HRNetV2-W18-Small-v2?

  2. Is "HRNetV2-W18-Small-v2" SOTA for lightweight segmentation? In terms of performance, HRNetV2-W18-Small-v2 is actually the best model, for now. But HRNetV2-W18-Small-v2 is not designed for lightweight segmentation, maybe speed is lower than U-HarDNet-70. My teammate has a new work based on HRNet for lightweight segmentation, and will release the code recently. You can check that.

  3. I am looking to implement high-quality semantic segmentation on a mobile device for accurate human segmentation for still images (i.e. non-realtime). Would "HRNetV2-W18-Small-v2" be a great option for this?

I think that HRNet is a good choice. In my opinion, HRNet achieves good performance on segmentation and human pose estimation, accurate prediction of keypoint should also help the human seg.

hsfzxjy commented 4 years ago

@InternetMaster1 I think you can implement HRNetV2-W18-Small-v2 + OCR by overriding MODEL.EXTRA in https://github.com/HRNet/HRNet-Semantic-Segmentation/blob/HRNet-OCR/experiments/cityscapes/seg_hrnet_ocr_w48_train_512x1024_sgd_lr1e-2_wd5e-4_bs_12_epoch484.yaml with the config of HRNetV2-W18-Small-v2.

You may also need to override MODEL.OCR.MID_CHANNELS and MODEL.OCR.KEY_CHANNELS. These two config items are not explicitly declared in .yaml file, but you can find them here. Practically, MODEL.OCR.MID_CHANNELS should be a bit lower than the final output channels of HRNet (default to 512 for HRNet-W48 with 720 channels), and KEY_CHANNELS be half of MID_CHANNELS. I suggest to set MODEL.OCR.MID_CHANNELS = 256 and MODEL.OCR.KEY_CHANNELS = 128.

SegFix is another standalone post-processing mechenism. You can find its usage in our openseg repository.

InternetMaster1 commented 4 years ago

@sunke123

Thank you for the plethora of information! Yes, HRNet looks like an awesome option for our needs!

1) > You can remove the sync-bn folder directly.

Can you be more specific? Along with deletion of the folder, do I need to even make any changes in any py files? Sorry for the newbie question, but I am stuck at installation

2) When is the new work based on HRNet for lightweight segmentation expected to release?

InternetMaster1 commented 4 years ago

@hsfzxjy

Many thanks for the detailed informtion. Once my installation is successful, I will check out the OCR option!

1) If you get a chance, could you add a yaml file for the same? Could be useful to other users too. Thanks.

2) Do you think using the OCR option will further improve the performance of HRNetV2-W18-Small-v2 and give a further better mIOU than the current mentioned 76.2?

3) What is the main purpose for SegFix? It is for smoothing of the edges?

sunke123 commented 4 years ago

@InternetMaster1

InplaceABN is not used in the code of the Pytorch v-1.1 branch. So, you can remove this folder without any changes.

The lightweight segmentation work is submitted to a CV conference. I think that they will release the code after the acceptance.

By the way, the author of lightweight hrnet tell me that you can try our HRNetV2-W18-Small-v2. It's actually a good choice.

InternetMaster1 commented 4 years ago

@sunke123

I was able to finally get it!

I was trying to use clone method from the pytorch-v1.1 branch. The URL for clone of the branch is same as the main repo, and hence the confusion! https://github.com/HRNet/HRNet-Semantic-Segmentation.git

I had to use the Download functionality to get the code for the branch https://github.com/HRNet/HRNet-Semantic-Segmentation/archive/pytorch-v1.1.zip

Thanks

InternetMaster1 commented 4 years ago

@hsfzxjy

If you get a chance, could you reply to the following, many thanks in advance

  1. Do you think using the OCR option will further improve the performance of HRNetV2-W18-Small-v2 and give a further better mIOU than the current mentioned 76.2?
  2. What is the main purpose for SegFix? It is for smoothing of the edges?
  3. If you get a chance, could you add a yaml file for HRNetV2-W18-Small-v2 + OCR combination? Could be useful to other users too. Thanks.
InternetMaster1 commented 4 years ago

@sunke123

How to convert the final model file to make it work on mobile, mainly Android?(i.e. tensorflow lite or ncnn). Are there any special steps involved?

I don't have any experience with HRNet and your guidance would be most helpful

Many thanks!

ywang370 commented 4 years ago

@InternetMaster1

InplaceABN is not used in the code of the Pytorch v-1.1 branch. So, you can remove this folder without any changes.

The lightweight segmentation work is submitted to a CV conference. I think that they will release the code after the acceptance.

By the way, the author of lightweight hrnet tell me that you can try our HRNetV2-W18-Small-v2. It's actually a good choice.

Hi I found 1.1 branch actually use batchnorm2d, I guess it is not syncnorm. Dose it mean no difference?

hsfzxjy commented 4 years ago

@InternetMaster1 InplaceABN is not used in the code of the Pytorch v-1.1 branch. So, you can remove this folder without any changes. The lightweight segmentation work is submitted to a CV conference. I think that they will release the code after the acceptance. By the way, the author of lightweight hrnet tell me that you can try our HRNetV2-W18-Small-v2. It's actually a good choice.

Hi I found 1.1 branch actually use batchnorm2d, I guess it is not syncnorm. Dose it mean no difference?

@ywang370 No. We use nn.SyncBatchNorm for PyTorch 1.1. See here.

sunke123 commented 4 years ago

@InternetMaster1

I also have no experience with the mobile application. Sorry for that

InternetMaster1 commented 4 years ago

Thank you @sunke123 & @hsfzxjy

You guys rock!