search

openPMD / openPMD-api

:floppy_disk: C++ & Python API for Scientific I/O
https://openpmd-api.readthedocs.io
GNU Lesser General Public License v3.0
142 stars 51 forks source link

`PYBIND11_FINDPYTHON=ON` #1684

Closed ax3l closed 1 month ago

ax3l commented 1 month ago

Reuse our find_package(Python ...) call and use new CMake logic in pybind11. https://pybind11.readthedocs.io/en/stable/compiling.html#modules-with-cmake https://cmake.org/cmake/help/latest/command/find_package.html#config-mode-version-selection

Fix https://github.com/openPMD/openPMD-api/pull/1677#issuecomment-2407743771

franzpoeschel commented 1 month ago

Should we add this logic to our superbuilds in CMake? Set PYTHON_EXECUTABLE before adding pybind11 if it is not yet set?

ax3l commented 1 month ago

Possibly, yes!

We should set the executable that was found here: https://github.com/openPMD/openPMD-api/blob/0.16.0/cmake/dependencies/pybind11.cmake#L80-L86

So a simple

set(PYTHON_EXECUTABLE ${Python_EXECUTABLE})

might make it already more robust.

ax3l commented 1 month ago

Ok, so if we set PYBIND11_FINDPYTHON=ON https://pybind11.readthedocs.io/en/stable/compiling.html#modules-with-cmake

Then pybind11 will call find_package(Python ...) again, which will see that we already called it (default find_package behavior), use our cached results and rely on the new logic https://github.com/pybind/pybind11/blob/v2.13.6/tools/pybind11Common.cmake#L215-L223

Well, not obvious from the docs, but ok.

ax3l commented 1 month ago

MSVC: calling setup.py with a python 3.9 now finds newest on the system first (outside of setup-python, venv and conda envs):

# python3.exe is c:\hostedtoolcache\windows\python\3.9.13\x64\python3.exe
$ python3.exe -m pip wheel -v .
...
  -- Found Python: C:/hostedtoolcache/windows/Python/3.12.7/x64/python3.exe (found suitable version "3.12.7", minimum required is "3.7.0") found components: Interpreter Development.Module
...

Which hints do we want to set in setup.py to make sure the calling Python process is exactly the Python that is used? (Note: Python_EXECUTABLE is an overwritable output, not an input. Setting it can still mismatch libs, etc.)

MSVC: NumPy requires newer Python:

  Link requires a different Python (3.9.13 not in: '>=3.10'): https://files.pythonhosted.org/packages/d1/0f/8d2b5ebb01dc49d20ae0a282d6baff7202b7bf0df8acdd4a6abeffe98070/numpy-2.1.0rc1.tar.gz (from https://pypi.org/simple/numpy/) (requires-python:>=3.10)
ax3l commented 1 month ago

Found a way to hint location and, if multiple versions are installed, version:

ax3l commented 1 month ago

Ok, that should do the trick :)

ax3l commented 1 month ago

Equivalent PRs to:

all pass :)