search

openvinotoolkit / openvino

OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference
https://docs.openvino.ai
Apache License 2.0
7.24k stars 2.26k forks source link

Build dldt 2020 with OMP on macOS Error #397

Closed qiyuangong closed 4 years ago

qiyuangong commented 4 years ago 
export CC=/usr/local/Cellar/llvm@7/7.1.0/bin/clang
export CXX=/usr/local/Cellar/llvm@7/7.1.0/bin/clang++
cmake -DCMAKE_BUILD_TYPE=Release -DTHREADING=OMP ..
macOS Catalina Version 10.15.3
[ 87%] Building CXX object inference-engine/src/vpu/graph_transformer/CMakeFiles/vpu_graph_transformer.dir/src/stages/post_op_stage.cpp.o
[ 87%] Linking CXX shared library ../../../../bin/intel64/Release/lib/libMKLDNNPlugin.dylib
Undefined symbols for architecture x86_64:
  "MKLDNNPlugin::cpu::getAvailableNUMANodes()", referenced from:
      MKLDNNPlugin::Config::readProperties(std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > > const&) in config.cpp.o
      MKLDNNPlugin::MKLDNNExecNetwork::MKLDNNExecNetwork(InferenceEngine::ICNNNetwork const&, MKLDNNPlugin::Config const&, std::__1::shared_ptr<MKLDNNPlugin::MKLDNNExtensionManager> const&) in mkldnn_exec_network.cpp.o
      create_shared_weights_per_socket() in mkldnn_plugin.cpp.o
ld: symbol(s) not found for architecture x86_64
clang-7: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [../bin/intel64/Release/lib/libMKLDNNPlugin.dylib] Error 1
make[1]: *** [inference-engine/src/mkldnn_plugin/CMakeFiles/MKLDNNPlugin.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 87%] Building CXX object ngraph/test/CMakeFiles/unit-test.dir/convert_u1_to_string.cpp.o
[ 87%] Building CXX object ngraph/test/CMakeFiles/unit-test.dir/coordinate.cpp.o
qiyuangong commented 4 years ago

This error makes building stop at 98% without libMKLDNNPlugin.dylib.

jgespino commented 4 years ago

Hi @qiyuangong

I am currently going through the build process, I will let you know if I succeed. In the meantime, could you try updating your clang version? The build instructions, list Clang compiler from Xcode 10.1 or higher as a software requirement.

Regards, Jesus

qiyuangong commented 4 years ago

Hi @jgespino

I have tried with higher Clang with latest Xcode. It encountered -fopenmp problem.

Here is the error message:

clang: error: unsupported option '-fopenmp'
make[2]: *** [inference-engine/src/preprocessing/CMakeFiles/inference_engine_preproc_obj.dir/cpu_x86_sse42/ie_preprocess_data_sse42.cpp.o] Error 1
make[1]: *** [inference-engine/src/preprocessing/CMakeFiles/inference_engine_preproc_obj.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[  1%] Building CXX object inference-engine/thirdparty/ade/sources/ade/CMakeFiles/ade.dir/source/assert.cpp.o
[  1%] Performing download step (git clone) for 'ext_gtest'
Cloning into 'ext_gtest'...
[  1%] Building CXX object inference-engine/thirdparty/ade/sources/ade/CMakeFiles/ade.dir/source/check_cycles.cpp.o
[  1%] Building CXX object inference-engine/thirdparty/ade/sources/ade/CMakeFiles/ade.dir/source/edge.cpp.o
[  1%] Building CXX object inference-engine/src/inference_engine/CMakeFiles/inference_engine_obj.dir/blob_factory.cpp.o
clang: error: unsupported option '-fopenmp'
make[2]: *** [inference-engine/src/inference_engine/CMakeFiles/inference_engine_obj.dir/blob_factory.cpp.o] Error 1
make[1]: *** [inference-engine/src/inference_engine/CMakeFiles/inference_engine_obj.dir/all] Error 2
[  1%] Building CXX object inference-engine/thirdparty/ade/sources/ade/CMakeFiles/ade.dir/source/execution_engine.cpp.o
[  2%] Building CXX object inference-engine/thirdparty/CMakeFiles/mkldnn.dir/mkl-dnn/src/common/batch_normalization.cpp.o
clang: error: unsupported option '-fopenmp'
make[2]: *** [inference-engine/thirdparty/CMakeFiles/mkldnn.dir/mkl-dnn/src/common/batch_normalization.cpp.o] Error 1
make[1]: *** [inference-engine/thirdparty/CMakeFiles/mkldnn.dir/all] Error 2
jgespino commented 4 years ago

Hi @qiyuangong

I ran into the same clang: error: unsupported option '-fopenmp' using the clang version included in xcode. However, I was able to get a successful build using the following steps. Could you give this a try and let me know if it works for you?

brew install llvm export CC=/usr/local/Cellar/llvm/9.0.1/bin/clang export CXX=/usr/local/Cellar/llvm/9.0.1/bin/clang++

git clone https://github.com/opencv/dldt.git cd dldt git submodule update --init --recursive mkdir build && cd build cmake -DCMAKE_BUILD_TYPE=Release -DTHREADING=OMP -DENABLE_OPENCV=OFF .. make -j 4

Regards, Jesus

qiyuangong commented 4 years ago

Hi @jgespino

I used your configuration. It still doesn't work for me. :(

Here is the error message in verbose build mode:

cmake -DCMAKE_BUILD_TYPE=Release -DTHREADING=OMP -DENABLE_OPENCV=OFF -DVERBOSE_BUILD=ON ..

[ 87%] Building CXX object ngraph/test/CMakeFiles/unit-test.dir/convert_u1_to_string.cpp.o
cd /Users/qiyuangong/Develop/dldt/build/ngraph/test && /usr/local/Cellar/llvm/9.0.1/bin/clang++  -DCURDIR=\"/Users/qiyuangong/Develop/dldt/ngraph/test\" -DENABLE_NGRAPH -DIE_BUILD_POSTFIX=\"\" -DJSON_INCLUDES=\"\" -DNGRAPH_JSON_DISABLE -DNGRAPH_UNIT_TEST_OPENVINO_ENABLE -DNGRAPH_VERSION=\"0.27.1-rc.0+b0bb801\" -DNGRAPH_VERSION_LABEL=\"v0.27.1-rc.0+b0bb801\" -DPROJECT_ROOT_DIR=\"/Users/qiyuangong/Develop/dldt/ngraph\" -DSERIALIZED_ZOO=\"/Users/qiyuangong/Develop/dldt/ngraph/test/models\" -DTEST_FILES=\"/Users/qiyuangong/Develop/dldt/ngraph/test/files\" -I/Users/qiyuangong/Develop/dldt/ngraph/test/. -I/Users/qiyuangong/Develop/dldt/inference-engine/include -I/Users/qiyuangong/Develop/dldt/ngraph/src -isystem /Users/qiyuangong/Develop/dldt/build/ngraph/gtest/src/ext_gtest/googletest/include -isystem /Users/qiyuangong/Develop/dldt/build/ngraph/gtest/src/ext_gtest/googlemock/include  -Wformat -Wformat-security -D_FORTIFY_SOURCE=2 -fstack-protector-all -fsigned-char -Werror -ffunction-sections -fdata-sections -fvisibility=hidden -fdiagnostics-show-option -Wundef -fvisibility-inlines-hidden -Wuninitialized -Winit-self -Wno-error=switch -Wno-error=uninitialized -Wno-error=literal-conversion -Wno-error=return-type -Wno-undef -fvisibility=default -DEIGEN_MPL2_ONLY -DTBB_USE_THREADING_TOOLS -O2 -Wformat -Wformat-security -D_FORTIFY_SOURCE=2 -fstack-protector-all -O3 -DNDEBUG -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -fPIE   -Wno-zero-as-null-pointer-constant -Wno-undef -Wno-reserved-id-macro -std=c++11 -o CMakeFiles/unit-test.dir/convert_u1_to_string.cpp.o -c /Users/qiyuangong/Develop/dldt/ngraph/test/convert_u1_to_string.cpp
Undefined symbols for architecture x86_64:
  "MKLDNNPlugin::cpu::getAvailableNUMANodes()", referenced from:
      MKLDNNPlugin::Config::readProperties(std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > > const&) in config.cpp.o
      MKLDNNPlugin::MKLDNNExecNetwork::MKLDNNExecNetwork(InferenceEngine::ICNNNetwork const&, MKLDNNPlugin::Config const&, std::__1::shared_ptr<MKLDNNPlugin::MKLDNNExtensionManager> const&) in mkldnn_exec_network.cpp.o
      create_shared_weights_per_socket() in mkldnn_plugin.cpp.o
ld: symbol(s) not found for architecture x86_64
clang-9: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [../bin/intel64/Release/lib/libMKLDNNPlugin.dylib] Error 1
make[1]: *** [inference-engine/src/mkldnn_plugin/CMakeFiles/MKLDNNPlugin.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

It seems that building [ 87%] Building CXX object ngraph/test/CMakeFiles/unit-test.dir/convert_u1_to_string.cpp.o encountered a problem.

Have a nice day! Qiyuan

qiyuangong commented 4 years ago

Hi @jgespino Which macOS version are you using? 10.14 or 10.15?

jgespino commented 4 years ago

Hi @qiyuangong

I tested on MacOS Catalina (10.15.3). When you followed by previous steps, did you start with a fresh build directory?

The MKLDNNPlugin.dir error was solved for me by adding the -DENABLE_OPENCV=OFF to my cmake command.

Regards, Jesus

qiyuangong commented 4 years ago

Hi @jgespino

I tried with fresh build directory (on both macOS 10.15 and macOS 10.12). They both works with TBB, but failed with OMP on the same place. :(

I'm trying to build with OMP on macOS 10.14 (fresh environment on my old macbook).

Regards, Qiyuan

qiyuangong commented 4 years ago

New update: Fresh dir with fresh build macOS 10.14 also doesn't work for OMP (both llvm and llvm@7). :(

jgespino commented 4 years ago

Hi @qiyuangong

I am re-installing macOS to try on a fresh build, I will let you know how it goes.

Regards, Jesus

qiyuangong commented 4 years ago

Thanks, Jesus. :)

jgespino commented 4 years ago

Hi @qiyuangong

I was able to successfully build dldt with a fresh macOS and installing the software versions listed below. Could you try to match my setup and give it another try?

Software versions: macOS Mojave (10.14.6) Python 3.6.5 CMake 3.16.5 Apple LLVM version 10.0.1 (clang-1001.0.46.4) libusb: stable 1.0.23

Commands: git clone https://github.com/opencv/dldt.git cd dldt/ git submodule update --init --recursive mkdir build cd build cmake -DCMAKE_BUILD_TYPE=Release -DTHREADING=OMP -DENABLE_OPENCV=OFF .. make -j 4

If you built fails again, please provide the full output of makecommand and the output of the following commands. clang --version python3 --version cmake --version git show -s (from the dldt directory) sw_vers

Regards, Jesus

alalek commented 4 years ago

@jgespino Please double check for typos, there is no "OMG" threading mode.

jgespino commented 4 years ago

Hi @alalek Ah yes good catch, I was typing too fast here!

jgespino commented 4 years ago

Hi @qiyuangong,

Turns out I had a typo in my cmake command as well, I re-started the rebuilt process and it failed with the same message you are seeing. I'm still looking into it as I may be missing some pre-requisites and the ./install_dependencies.sh doesn't work for macOS (probably a bug).

@alalek Have you had a successful built on MacOS with OMP?

Regards, Jesus

qiyuangong commented 4 years ago

Thanks @jgespino !

Hi @alalek

According to my building tests, TBB option passed on all platform. But, for OMP, there are still some small problems, such as OpenCV on CentOS, and this macOS building issue etc.

Is there any hot-fix for this problem?

Regards, Qiyuan

qiyuangong commented 4 years ago

Hi @jgespino

I found the root cause in https://github.com/opencv/dldt/blob/2020/inference-engine/src/mkldnn_plugin/mkldnn/system_conf.cpp#L46.

getAvailableNUMANodes is only defined for TBB on macOS.

That's why this issue only happened on macOS with OMP. :(

qiyuangong commented 4 years ago

Fix build by adding these lines to system_conf.cpp.

#if !((IE_THREAD == IE_THREAD_TBB || IE_THREAD == IE_THREAD_TBB_AUTO))
 std::vector<int> getAvailableNUMANodes() {
        std::vector<int> numa_indexes;
        numa_indexes.push_back(0);
        return numa_indexes;
    }
#endif

Because recent mac products only have one CPU. That makes it possible to hard code 0 for return value. :)

Regards, Qiyuan

qiyuangong commented 4 years ago

Problem solved. Issue closed.

Thank you @jgespino ! :)