NVIDIA / MinkowskiEngine

Minkowski Engine is an auto-diff neural network library for high-dimensional sparse tensors
https://nvidia.github.io/MinkowskiEngine
Other
2.43k stars 360 forks source link

Build Successful, Import fails, dlopen - symbol not found in flat namespace '___kmpc_for_static_fini' #505

Open kghandour opened 1 year ago

kghandour commented 1 year ago

Describe the bug After building from source on an M1 Pro Macbook Pro, with parameters --cpu_only --blas=openblas, importing MinkowskiEngine fails with the error mentioned above.


To Reproduce Steps to reproduce the behavior. If the code is not attached and cannot be reproduced easily, the bug report will be closed without any comments.

python3 setup.py install --cpu_only --blas=openblas 

Expected behavior For MinkowskiEngine to get imported properly.


Desktop (please complete the following information):

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/kghandour/MinkowskiEngine/MinkowskiEngine/__init__.py", line 53, in <module>
    from MinkowskiEngineBackend._C import (
ImportError: dlopen(/opt/anaconda3/envs/perfception/lib/python3.8/site-packages/MinkowskiEngine-0.5.4-py3.8-macosx-11.1-arm64.egg/MinkowskiEngineBackend/_C.cpython-38-darwin.so, 0x0002): symbol not found in flat namespace '___kmpc_for_static_fini'

Additional context From my research I have found that the issue involves libomp not being linked properly, so I have done the following:

  1. I am using LLVM installed from brew, exported CC to LLVM clang,
  2. I am using openblas installed from brew
  3. I am using libomp installed from brew
  4. I am using gcc installed from brew

additionally have these set in my ~/.zshrc

export LDFLAGS="-L/opt/homebrew/opt/llvm/lib -L/opt/homebrew/opt/openblas/lib -L/opt/homebrew/opt/libomp/lib"
export CPPFLAGS="-I/opt/homebrew/opt/llvm/include -I/opt/homebrew/opt/openblas/include -I/opt/homebrew/opt/libomp/include"
export PKG_CONFIG_PATH="/opt/homebrew/opt/openblas/lib/pkgconfig"
export BLAS=/opt/homebrew/opt/openblas/lib/libopenblasp-r0.3.21.dylib
alias clang='/opt/homebrew/opt/llvm/bin/clang'
alias gcc='/opt/homebrew/Cellar/gcc/12.2.0/bin/gcc-12'
alias g++='/opt/homebrew/Cellar/gcc/12.2.0/bin/g++-12'
export DYLD_LIBRARY_PATH="/opt/homebrew/opt/libomp/lib:$DYLD_LIBRARY_PATH"
export CC="/opt/homebrew/opt/llvm/bin/clang"
  1. I have tried to modify the setup.py script even with the following changes:
extra_link_args = ['-lomp']
include_dirs = ['/opt/homebrew/opt/libomp/include']
CC_FLAGS += ["-Xpreprocessor", "-fopenmp"]

Build successful, but the import fails.

Installed /opt/anaconda3/envs/perfception/lib/python3.8/site-packages/MinkowskiEngine-0.5.4-py3.8-macosx-11.1-arm64.egg
.
.
.
Finished processing dependencies for MinkowskiEngine==0.5.4

Please note that I am trying different things, and once I zone in on the issue, I can clean up. I have also found https://github.com/NVIDIA/MinkowskiEngine/pull/438, that talks about a similar issue with building on darwin, however, from my research I cannot use mkl on an M1 Mac so I am using openblas.

Thank you for any help you can provide.