I need to use sentencepiece for tokenization, and I also need OpenVINO for NLP task inference. I am using vcpkg to manage both sentencepiece and OpenVINO. The protobuf for OpenVINO comes from vcpkg, while the protobuf for sentencepiece defaults to internal rather than package. When I link both OpenVINO and sentencepiece libraries, I encounter symbol redefinition issues. Therefore, I have to change the SPM_PROTOBUF_PROVIDER of sentencepiece to "package" to use protobuf from vcpkg. However, after making this change, the compilation fails. How can I resolve this issue?
cmake change
@@ -33,7 +33,7 @@ option(SPM_NO_THREADLOCAL "Disable thread_local operator" OFF)
option(SPM_ENABLE_MSVC_MT_BUILD, "Use /MT flag in MSVC build" OFF)
option(SPM_CROSS_SYSTEM_PROCESSOR, "Override system processor" "")
-set(SPM_PROTOBUF_PROVIDER "internal" CACHE STRING "Provider of protobuf library")
+set(SPM_PROTOBUF_PROVIDER "package" CACHE STRING "Provider of protobuf library")
set_property(CACHE SPM_PROTOBUF_PROVIDER PROPERTY STRINGS "internal" "package")
set(SPM_ABSL_PROVIDER "internal" CACHE STRING "Provider of absl library")
set_property(CACHE SPM_ABSL_PROVIDER PROPERTY STRINGS "internal" "module" "package")
log
···
[build] C:\PROGRA1\MICROS1\2022\COMMUN1\VC\Tools\MSVC\14401.338\bin\Hostx86\x64\cl.exe /nologo /TP -D_USE_EXTERNAL_PROTOBUF -D_USE_INTERNAL_STRING_VIEW -IH:\cpp_faster\sentencepiece -IH:\cpp_faster\sentencepiece\out\build\win-x64-release -IH:\cpp_faster\sentencepiece\third_party\absl -IH:\vcpkg\vcpkg_win\installed\x64-windows-static\include -IH:\cpp_faster\sentencepiece\out\build\win-x64-release\src -IH:\cpp_faster\sentencepiece\src..\third_party /DWIN32 /D_WINDOWS /W3 /GR /EHsc /O2 /Ob2 /DNDEBUG -std
++17 -MD /wd4267 /wd4244 /wd4305 /Zc
/utf-8 /showIncludes /Fosrc\CMakeFiles\sentencepiece-static.dir\error.cc.obj /Fdsrc\CMakeFiles\sentencepiece-static.dir\sentencepiece-static.pdb /FS -c H:\cpp_faster\sentencepiece\src\error.cc
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/casts.h(93): error C2872: “internal”: ambiguous symbol
[build] H:\cpp_faster\sentencepiece\third_party/absl/flags/flag.h(24): note: could be “absl::internal”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/internal/identity.h(23): note: or “absl::lts_20240116::internal”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/casts.h(93): error C2039: "type_identity_t": is not a member of "absl::internal"
[build] H:\cpp_faster\sentencepiece\third_party/absl/flags/flag.h(24): note: see declaration of "absl::internal"
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/casts.h(93): error C2988: unrecognizable template declaration/definition
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/casts.h(93): error C2143: syntax error: missing “,” before “<”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/casts.h(93): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/utility/utility.h(94): error C2872: “internal”: ambiguous symbol
[build] H:\cpp_faster\sentencepiece\third_party/absl/flags/flag.h(24): note: could be “absl::internal”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/internal/identity.h(23): note: or “absl::lts_20240116::internal”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/utility/utility.h(94): error C2039: "type_identity_t": is not a member of "absl::internal"
[build] H:\cpp_faster\sentencepiece\third_party/absl/flags/flag.h(24): note: see declaration of "absl::internal"
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/utility/utility.h(94): error C2143: syntax error: missing “;” before “<”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/utility/utility.h(94): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/utility/utility.h(94): error C2737: “absl::lts_20240116::type_identity_t”: constexpr object must be initialized
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/utility/utility.h(94): error C2143: syntax error: missing “;” before “{”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/utility/utility.h(94): error C2447: “{”: missing function header (old-style formal list?)
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(744): error C2872: “internal”: ambiguous symbol
[build] H:\cpp_faster\sentencepiece\third_party/absl/flags/flag.h(24): note: could be “absl::internal”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/internal/identity.h(23): note: or “absl::lts_20240116::internal”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(744): error C2039: "type_identity": is not a member of "absl::internal"
[build] H:\cpp_faster\sentencepiece\third_party/absl/flags/flag.h(24): note: see declaration of "absl::internal"
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(744): error C2988: unrecognizable template declaration/definition
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(744): error C2143: syntax error: missing “,” before “<”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(744): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(756): error C2872: “internal”: ambiguous symbol
[build] H:\cpp_faster\sentencepiece\third_party/absl/flags/flag.h(24): note: could be “absl::internal”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/internal/identity.h(23): note: or “absl::lts_20240116::internal”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(756): error C2039: "type_identity": is not a member of "absl::internal"
[build] H:\cpp_faster\sentencepiece\third_party/absl/flags/flag.h(24): note: see declaration of "absl::internal"
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(756): error C2988: unrecognizable template declaration/definition
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(756): error C2143: syntax error: missing “)” before “<”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(756): error C2143: syntax error: missing “,” before “<”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(756): error C2059: syntax error: “)”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(755): error C2091: function returns function
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(761): error C2872: “internal”: ambiguous symbol
[build] H:\cpp_faster\sentencepiece\third_party/absl/flags/flag.h(24): note: could be “absl::internal”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/internal/identity.h(23): note: or “absl::lts_20240116::internal”
···
C:\PROGRA~1\MICROS~1\2022\COMMUN~1\VC\Tools\MSVC\1440~1.338\bin\Hostx64\x64\cl.exe /TP -D_USE_EXTERNAL_PROTOBUF -D_USE_INTERNAL_STRING_VIEW -IH:\vcpkg\vcpkg_win\buildtrees\sentencepiece\src\v0.1.99-67e41e1247.clean -IH:\vcpkg\vcpkg_win\buildtrees\sentencepiece\x64-windows-static-dbg -IH:\vcpkg\vcpkg_win\installed\x64-windows-static\include -IH:\vcpkg\vcpkg_win\buildtrees\sentencepiece\x64-windows-static-dbg\src -IH:\vcpkg\vcpkg_win\buildtrees\sentencepiece\src\v0.1.99-67e41e1247.clean\src\..\third_party /nologo /DWIN32 /D_WINDOWS /W3 /utf-8 /GR /EHsc /MP /D_DEBUG /MTd /Z7 /Ob0 /Od /RTC1 -std:c++17 -MTd /wd4267 /wd4244 /wd4305 /Zc:strictStrings /utf-8 /showIncludes /Fosrc\CMakeFiles\sentencepiece_train-static.dir\unigram_model_trainer.cc.obj /Fdsrc\CMakeFiles\sentencepiece_train-static.dir\sentencepiece_train-static.pdb /FS -c H:\vcpkg\vcpkg_win\buildtrees\sentencepiece\src\v0.1.99-67e41e1247.clean\src\unigram_model_trainer.cc
H:\vcpkg\vcpkg_win\buildtrees\sentencepiece\src\v0.1.99-67e41e1247.clean\src\sentencepiece_processor.h(58): error C2872: 'string_view': ambiguous symbol
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\include\xstring(1788): note: could be 'std::basic_string_view<char,std::char_traits<char>> std::string_view'
H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/strings/string_view.h(162): note: or 'absl::lts_20240116::string_view'
H:\vcpkg\vcpkg_win\buildtrees\sentencepiece\src\v0.1.99-67e41e1247.clean\src\sentencepiece_processor.h(244): error C2872: 'string_view': ambiguous symbol
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\include\xstring(1788): note: could be 'std::basic_string_view<char,std::char_traits<char>> std::string_view'
H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/strings/string_view.h(162): note: or 'absl::lts_20240116::string_view'
H:\vcpkg\vcpkg_win\buildtrees\sentencepiece\src\v0.1.99-67e41e1247.clean\src\sentencepiece_processor.h(248): error C2872: 'string_view': ambiguous symbol
···
I need to use sentencepiece for tokenization, and I also need OpenVINO for NLP task inference. I am using vcpkg to manage both sentencepiece and OpenVINO. The protobuf for OpenVINO comes from vcpkg, while the protobuf for sentencepiece defaults to internal rather than package. When I link both OpenVINO and sentencepiece libraries, I encounter symbol redefinition issues. Therefore, I have to change the SPM_PROTOBUF_PROVIDER of sentencepiece to "package" to use protobuf from vcpkg. However, after making this change, the compilation fails. How can I resolve this issue?
cmake change
log