Build fails with Python 3.11 because of outdated pybind11

thesamesam commented 1 year ago

Describe the bug

Building tweedledum-1.1.1 with Python 3.11 fails as follows:

[...]
/var/tmp/portage/dev-python/tweedledum-1.1.1-r1/work/tweedledum-1.1.1/external/pybind11/include/pybind11/pybind11.h:2296:18: error: invalid use of incomplete type ‘PyFrameObject’ {aka ‘struct _frame’}
 2296 |             frame->f_locals, PyTuple_GET_ITEM(frame->f_code->co_varnames, 0));
      |                  ^~
/usr/include/python3.11/pytypedefs.h:22:16: note: forward declaration of ‘PyFrameObject’ {aka ‘struct _frame’}
   22 | typedef struct _frame PyFrameObject;
      |                ^~~~~~
In file included from /usr/include/python3.11/Python.h:38:
/var/tmp/portage/dev-python/tweedledum-1.1.1-r1/work/tweedledum-1.1.1/external/pybind11/include/pybind11/pybind11.h:2296:30: error: invalid use of incomplete type ‘PyFrameObject’ {aka ‘struct _frame’}
 2296 |             frame->f_locals, PyTuple_GET_ITEM(frame->f_code->co_varnames, 0));
      |                              ^~~~~~~~~~~~~~~~
/usr/include/python3.11/pytypedefs.h:22:16: note: forward declaration of ‘PyFrameObject’ {aka ‘struct _frame’}
   22 | typedef struct _frame PyFrameObject;
      |                ^~~~~~
[30/73] Building CXX object CMakeFiles/_tweedledum.dir/src/Decomposition/BarencoDecomposer.cpp.o
[31/73] Building CXX object CMakeFiles/_tweedledum.dir/src/Parser/QASM/Parser.cpp.o
[32/73] Building CXX object CMakeFiles/_tweedledum.dir/src/Passes/Mapping/Placer/ApprxSatPlacer.cpp.o
[33/73] Building CXX object CMakeFiles/_tweedledum.dir/src/Passes/Mapping/Placer/SatPlacer.cpp.o
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/skbuild/setuptools_wrap.py", line 642, in setup
    cmkr.make(make_args, install_target=cmake_install_target, env=env)
  File "/usr/lib/python3.11/site-packages/skbuild/cmaker.py", line 679, in make
    self.make_impl(clargs=clargs, config=config, source_dir=source_dir, install_target=install_target, env=env)
  File "/usr/lib/python3.11/site-packages/skbuild/cmaker.py", line 710, in make_impl
    raise SKBuildError(

An error occurred while building with CMake.
  Command:
    /usr/bin/cmake --build . --target install --config Release -- -j 32
  Install target:
    install
  Source directory:
    /var/tmp/portage/dev-python/tweedledum-1.1.1-r1/work/tweedledum-1.1.1
  Working directory:
    /var/tmp/portage/dev-python/tweedledum-1.1.1-r1/work/tweedledum-1.1.1/_skbuild/linux-x86_64-3.11/cmake-build
Please check the install target is valid and see CMake's output for more information.
[EE] ERROR: dev-python/tweedledum-1.1.1-r1::gentoo failed (compile phase):

I think this is because the bundled copy of pybind11 is a bit stale.

Reproducible example code

/usr/bin/cmake /var/tmp/portage/dev-python/tweedledum-1.1.1-r1/work/tweedledum-1.1.1 -G Ninja -DCMAKE_INSTALL_PREFIX:PATH=/var/tmp/portage/dev-python/tweedledum-1.1.1-r1/work/tweedledum-1.1.1/_skbuild/linux-x86_64-3.11/cmake-install/python -DPYTHON_VERSION_STRING:STRING=3.11.2 -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/usr/lib/python3.11/site-packages/skbuild/resources/cmake -DPYTHON_EXECUTABLE:PATH=/usr/bin/python3.11 -DPYTHON_INCLUDE_DIR:PATH=/usr/include/python3.11 -DPYTHON_LIBRARY:PATH=/usr/lib64/libpython3.11.so -DPython_EXECUTABLE:PATH=/usr/bin/python3.11 -DPython_ROOT_DIR:PATH=/usr -DPython_INCLUDE_DIR:PATH=/usr/include/python3.11 -DPython_FIND_REGISTRY:STRING=NEVER -DPython_NumPy_INCLUDE_DIRS:PATH=/usr/lib/python3.11/site-packages/numpy/core/include -DPython3_EXECUTABLE:PATH=/usr/bin/python3.11 -DPython3_ROOT_DIR:PATH=/usr -DPython3_INCLUDE_DIR:PATH=/usr/include/python3.11 -DPython3_FIND_REGISTRY:STRING=NEVER -DPython3_NumPy_INCLUDE_DIRS:PATH=/usr/lib/python3.11/site-packages/numpy/core/include -DCMAKE_BUILD_TYPE:STRING=Release -DCMAKE_BUILD_TYPE=RelWithDebInfo
/usr/bin/cmake --build . --target install --config Release -- -j 32

Expected behavior

Successful build.

Information

tweedledum version: 1.1.1
Python version: Python 3.11.2
C++ compiler: GCC 12.2.1_p20230218
Operating system: Gentoo

iyanmv commented 1 year ago

I can confirm the issue after replacing external/pybind11 with version 2.10.4. Also, it would be nice if CMakeLists.txt included the option to find a local installed version. The option TWEEDLEDUM_USE_EXTERNAL_PYBIND11 doesn't work as expected for me. Also, for recent versions of gcc, you have to add #include <cstdint> to Cbit.h. This needs some attention, as latest release was already broken (and had to manually applied #170), but now it just got worse after many distros updating to Python 3.11 and GCC 13.

iyanmv commented 1 year ago

Quick fix:

cd external
rm -r pybind11/*
curl -OLs https://github.com/pybind/pybind11/archive/refs/tags/v2.10.4.tar.gz
tar -xzf v2.10.4.tar.gz --strip-components=1 --directory=pybind11
cd ../include/tweedledum/IR
sed -i '6s/.*/#include <cstdint>/' Cbit.h

boschmitt / tweedledum