search

SHI-Labs / OneFormer

OneFormer: One Transformer to Rule Universal Image Segmentation, arxiv 2022 / CVPR 2023
https://praeclarumjj3.github.io/oneformer
MIT License
1.39k stars 129 forks source link

undefined symbol: _Z27ms_deform_attn_cuda_forwardRKN2at6TensorES2_S2_S2_S2_i #81

Closed Omeracan closed 11 months ago

Omeracan commented 1 year ago

Hello,

I try to following the training instruction and run this command

python train_net.py --dist-url 'tcp://127.0.0.1:50163' \
--num-gpus 4 \
--config-file /home/cplabpla/OneFormer/configs/cityscapes/oneformer_R50_bs16_90k.yaml \
OUTPUT_DIR /home/cplabpla/OneFormer/outputs/cityscapes_R50_bs16_90k_test_1 \
WANDB.NAME cityscapes_R50_bs16_90k_test_1

  File "train_net.py", line 43, in <module>
    from oneformer.evaluation import (
  File "/lustrefs/disk/home/cplabpla/OneFormer/oneformer/__init__.py", line 2, in <module>
    from . import modeling
  File "/lustrefs/disk/home/cplabpla/OneFormer/oneformer/modeling/__init__.py", line 5, in <module>
    from .pixel_decoder.msdeformattn import MSDeformAttnPixelDecoder
  File "/lustrefs/disk/home/cplabpla/OneFormer/oneformer/modeling/pixel_decoder/msdeformattn.py", line 23, in <module>
    from .ops.modules import MSDeformAttn
  File "/lustrefs/disk/home/cplabpla/OneFormer/oneformer/modeling/pixel_decoder/ops/modules/__init__.py", line 10, in <module>
    from .ms_deform_attn import MSDeformAttn
  File "/lustrefs/disk/home/cplabpla/OneFormer/oneformer/modeling/pixel_decoder/ops/modules/ms_deform_attn.py", line 25, in <module>
    from ..functions import MSDeformAttnFunction
  File "/lustrefs/disk/home/cplabpla/OneFormer/oneformer/modeling/pixel_decoder/ops/functions/__init__.py", line 9, in <module>
    from .ms_deform_attn_func import MSDeformAttnFunction
  File "/lustrefs/disk/home/cplabpla/OneFormer/oneformer/modeling/pixel_decoder/ops/functions/ms_deform_attn_func.py", line 23, in <module>
    import MultiScaleDeformableAttention as MSDA
ImportError: /home/cplabpla/.conda/envs/oneformer/lib/python3.8/site-packages/MultiScaleDeformableAttention-1.0-py3.8-linux-x86_64.egg/MultiScaleDeformableAttention.cpython-38-x86_64-linux-gnu.so: undefined symbol: _Z27ms_deform_attn_cuda_forwardRKN2at6TensorES2_S2_S2_S2_i

I was following the Installation instruction https://github.com/SHI-Labs/OneFormer/blob/main/INSTALL.md At Setup CUDA Kernel for MSDeformAttn step, It shows

running build
running build_py
running build_ext
running install
running bdist_egg
running egg_info
writing MultiScaleDeformableAttention.egg-info/PKG-INFO
writing dependency_links to MultiScaleDeformableAttention.egg-info/dependency_links.txt
writing top-level names to MultiScaleDeformableAttention.egg-info/top_level.txt
reading manifest file 'MultiScaleDeformableAttention.egg-info/SOURCES.txt'
writing manifest file 'MultiScaleDeformableAttention.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/modules
copying build/lib.linux-x86_64-3.8/modules/__init__.py -> build/bdist.linux-x86_64/egg/modules
copying build/lib.linux-x86_64-3.8/modules/ms_deform_attn.py -> build/bdist.linux-x86_64/egg/modules
creating build/bdist.linux-x86_64/egg/functions
copying build/lib.linux-x86_64-3.8/functions/__init__.py -> build/bdist.linux-x86_64/egg/functions
copying build/lib.linux-x86_64-3.8/functions/ms_deform_attn_func.py -> build/bdist.linux-x86_64/egg/functions
copying build/lib.linux-x86_64-3.8/MultiScaleDeformableAttention.cpython-38-x86_64-linux-gnu.so -> build/bdist.linux-x86_64/egg
byte-compiling build/bdist.linux-x86_64/egg/modules/__init__.py to __init__.cpython-38.pyc
byte-compiling build/bdist.linux-x86_64/egg/modules/ms_deform_attn.py to ms_deform_attn.cpython-38.pyc
byte-compiling build/bdist.linux-x86_64/egg/functions/__init__.py to __init__.cpython-38.pyc
byte-compiling build/bdist.linux-x86_64/egg/functions/ms_deform_attn_func.py to ms_deform_attn_func.cpython-38.pyc
creating stub loader for MultiScaleDeformableAttention.cpython-38-x86_64-linux-gnu.so
byte-compiling build/bdist.linux-x86_64/egg/MultiScaleDeformableAttention.py to MultiScaleDeformableAttention.cpython-38.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying MultiScaleDeformableAttention.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying MultiScaleDeformableAttention.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying MultiScaleDeformableAttention.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying MultiScaleDeformableAttention.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
writing build/bdist.linux-x86_64/egg/EGG-INFO/native_libs.txt
/home/cplabpla/.conda/envs/oneformer/lib/python3.8/site-packages/torch/utils/cpp_extension.py:381: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
  warnings.warn(msg.format('we could not find ninja.'))
/home/cplabpla/.conda/envs/oneformer/lib/python3.8/site-packages/torch/utils/cpp_extension.py:782: UserWarning: The detected CUDA version (11.7) has a minor version mismatch with the version that was used to compile PyTorch (11.3). Most likely this shouldn't be a problem.
  warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda))
/home/cplabpla/.conda/envs/oneformer/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/home/cplabpla/.conda/envs/oneformer/lib/python3.8/site-packages/setuptools/command/easy_install.py:156: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
zip_safe flag not set; analyzing archive contents...
__pycache__.MultiScaleDeformableAttention.cpython-38: module references __file__
creating 'dist/MultiScaleDeformableAttention-1.0-py3.8-linux-x86_64.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing MultiScaleDeformableAttention-1.0-py3.8-linux-x86_64.egg
creating /lustrefs/disk/home/cplabpla/.conda/envs/oneformer/lib/python3.8/site-packages/MultiScaleDeformableAttention-1.0-py3.8-linux-x86_64.egg
Extracting MultiScaleDeformableAttention-1.0-py3.8-linux-x86_64.egg to /lustrefs/disk/home/cplabpla/.conda/envs/oneformer/lib/python3.8/site-packages
Adding MultiScaleDeformableAttention 1.0 to easy-install.pth file

Not sure, the installation is completely success. Does anybody know how can I solve this?

Thank you

praeclarumjj3 commented 12 months ago

Hi @Omeracan, did you retry installing MSDeformAttn?

Omeracan commented 12 months ago

Sorry for the late reply @praeclarumjj3 , I try to re-installing MSDeformAttn but the result has the same as I mention above.

Omeracan commented 11 months ago

The solution: Change natten lib compact to your CUDA version.