search

microsoft / vcpkg

C++ Library Manager for Windows, Linux, and MacOS
MIT License
22.18k stars 6.16k forks source link

[pybind11] Fail to link debug python library #37380

Open zyrkiel opened 3 months ago

zyrkiel commented 3 months ago

Describe the bug While linking to a custom library depending on pybind11 in Debug I get an undefined reference. It is due to the fact that the release Python3 lib is given to the link instead of the debug one.
This issue might be related to #33724

Environment

To Reproduce Code is available here: https://github.com/zyrkiel/test_link_python_debug

Steps to reproduce the behavior:

  1. git clone --recurse-submodules https://github.com/zyrkiel/test_link_python_debug.git
  2. cd test_link_python_debug
  3. cmake -DBUILD_SHARED_LIBS:BOOL=OFF -DCMAKE_TOOLCHAIN_FILE=vcpkg/scripts/buildsystems/vcpkg.cmake -DVCPKG_MANIFEST_INSTALL:BOOL=ON -DVCPKG_HOST_TRIPLET=x64-linux -DVCPKG_TARGET_TRIPLET=x64-linux -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_BUILD_TYPE:STRING=Debug -DCMAKE_INSTALL_PREFIX:STRING=install -DCMAKE_EXPORT_COMPILE_COMMANDS:BOOL=TRUE -S . -B build
  4. cmake --build build --config Debug

However, running again the configuration and the build will lead to a success.

Expected behavior cmake should find the debug library of python and links it

Failure logs

First Configure output:

-- Found pybind11: /home/tgeffroy/GitRepo/git/test_link_python_debug/build/vcpkg_installed/x64-linux/include (found version "2.11.1")
-- Found Python3: /home/tgeffroy/GitRepo/git/test_link_python_debug/build/vcpkg_installed/x64-linux/tools/python3/python3.11 (found version "3.11.5") found components: Interpreter Development Development.Module Development.Embed

First Build output:

[...]
[100%] Linking CXX executable PythonLinkTest
/usr/local/bin/cmake -E cmake_link_script CMakeFiles/PythonLinkTest.dir/link.txt --verbose=1
/usr/bin/c++ -g CMakeFiles/PythonLinkTest.dir/main.cpp.o -o PythonLinkTest  **foo_lib/libFoo.a vcpkg_installed/x64-linux/lib/libpython3.11.a** -ldl 
/usr/bin/ld: foo_lib/libFoo.a(foo.cpp.o): in function `Py_INCREF':
/home/tgeffroy/GitRepo/git/test_link_python_debug/build/vcpkg_installed/x64-linux/include/python3.11/object.h:500: undefined reference to `_Py_RefTotal'
/usr/bin/ld: /home/tgeffroy/GitRepo/git/test_link_python_debug/build/vcpkg_installed/x64-linux/include/python3.11/object.h:500: undefined reference to `_Py_RefTotal'
/usr/bin/ld: foo_lib/libFoo.a(foo.cpp.o): in function `Py_DECREF':
/home/tgeffroy/GitRepo/git/test_link_python_debug/build/vcpkg_installed/x64-linux/include/python3.11/object.h:520: undefined reference to `_Py_RefTotal'
/usr/bin/ld: /home/tgeffroy/GitRepo/git/test_link_python_debug/build/vcpkg_installed/x64-linux/include/python3.11/object.h:520: undefined reference to `_Py_RefTotal'
/usr/bin/ld: /home/tgeffroy/GitRepo/git/test_link_python_debug/build/vcpkg_installed/x64-linux/include/python3.11/object.h:523: undefined reference to `_Py_NegativeRefcount'
[...]

Second Configure output:

-- Found pybind11: /home/tgeffroy/GitRepo/git/test_link_python_debug/build/vcpkg_installed/x64-linux/include (found version "2.11.1")
-- Found Python3: /home/tgeffroy/GitRepo/git/test_link_python_debug/build/vcpkg_installed/x64-linux/tools/python3/python3.11 (found version "3.11.5") found components: Interpreter Development Development.Module Development.Embed

Second Build output:

[ 75%] Linking CXX executable PythonLinkTest
/usr/local/bin/cmake -E cmake_link_script CMakeFiles/PythonLinkTest.dir/link.txt --verbose=1
/usr/bin/c++ -g CMakeFiles/PythonLinkTest.dir/main.cpp.o -o PythonLinkTest  foo_lib/libFoo.a vcpkg_installed/x64-linux/debug/lib/libpython3.11d.a -ldl 
gmake[2]: Leaving directory '/home/tgeffroy/GitRepo/git/test_link_python_debug/build'
[100%] Built target PythonLinkTest

Additional context The issue is not dependent from compiler, nor generator. As you can see, the link is trying to use the release lib foo_lib/libFoo.a vcpkg_installed/x64-linux/lib/libpython3.11.a

The proposed solution to #33724 isn't working: set(Python_FIND_ABI "ON" "ANY" "ANY") before trying to find packages. It gives the same behavior.

zyrkiel commented 3 months ago

It seems more logic to find python3 before pybind11. So I have switched the 2 lines.

After that, I have realized that I should use Python3_FIND_ABI instead of Python_FIND_ABI since I am trying to find Python3.

Finally, removing Python3 from the vcpkg.json dependencies enables the configure to end successfully and to link to local Python.

You can find the workaround.patch in the repository.

However, I still would like to use vcpkg python3 package and not to specify the build type at the configure phase.