tttapa / py-build-cmake

Modern, PEP 517 compliant build backend for creating Python packages with extensions built using CMake.
https://pypi.org/project/py-build-cmake
MIT License
38 stars 6 forks source link

Q: how to automatically detect platform_tag? #29

Open ilya-lavrenov opened 3 weeks ago

ilya-lavrenov commented 3 weeks ago

Question 1: Currently, when I built wheel on Linux, platform tag is something like linux_x86_64 How to ensure that it detect current glibc version and set platform tag in manylinix_x_y_arch format?

I don't want to use auditwheel for it, because it's an extra step and not user friendly.

Question 2: how to specify that my project does not contain Python extension and ensure py3-none is used instead of cp310-cp310

Question 3: How to install several license files / custom license to the folder dist-info?

ilya-lavrenov commented 3 weeks ago

For questions 1 and 2 I managed to partially WA the issue with:

[tool.py-build-cmake.cross]
implementation = "py"
version = "3"
abi = "none"
arch = "manylinux_2_31_x86_64"
toolchain_file = "cmake/native.cmake"

with empty cmake/native.cmake.

But I still have to hardcode manylinux_2_31_x86_64 which can be different depending on glibc version of specific Linux OS

tttapa commented 3 weeks ago

Question 1

I believe there are three common scenarios here:

  1. You (or a user) are building packages from source locally. In this case, py-build-cmake simply uses the current interpreter's platform tag (as provided by distlib, e.g. linux_x86_64). This is okay for local packages, but these packages should never be uploaded to PyPI.
  2. You're building a package for distribution on PyPI. This should really only be done in a CI/CD workflow, where you use standard tools such as cibuildwheel, which will process your package using auditwheel for you. This is necessary to make sure you don't upload incompatible packages with broken dependencies. Since this only happens in a CI workflow, you don't have to worry about user friendliness.
  3. You're cross-compiling. Here, you should know the glibc version of the toolchain you're using, and you can indeed use the cross.arch tag for this purpose. (In fact, you cannot even use auditwheel in this case.) If you use an appropriate toolchain and you handle dependencies carefully, these packages can also be uploaded to PyPI.

I'm open to adding an arch option to native builds, but IMHO, py-build-cmake should not try to deduce the glibc version automatically, because A. this is unnecessary for local builds, and B. it may result in people uploading these local wheels to PyPI by accident.

Question 2

Using the py3-none tags is not yet supported by the stable release. In the current pre-release I'm working on, you can set python_abi = 'none' in your [tool.py-build-cmake.cmake] config, see https://github.com/tttapa/py-build-cmake/blob/rework-0.2.0/docs/Config.md.

Question 3

You should be able to simply install them using an install(FILES ...) command in your CMakeLists.txt. For instance:

install(FILES LICENSE.custom
        EXCLUDE_FROM_ALL
        COMPONENT python_modules
        DESTINATION "${PY_BUILD_CMAKE_PACKAGE_NAME}-${PY_BUILD_CMAKE_PACKAGE_VERSION}.dist-info")
ilya-lavrenov commented 3 weeks ago

Hi @tttapa

Thank you for prompt response and for the project itself. I really find it very useful!

I'm open to adding an arch option to native builds, but IMHO, py-build-cmake should not try to deduce the glibc version automatically, because A. this is unnecessary for local builds, and B. it may result in people uploading these local wheels to PyPI by accident.

I'm open to adding an arch option to native builds, but IMHO, py-build-cmake should not try to deduce the glibc version automatically, because A. this is unnecessary for local builds, and B. it may result in people uploading these local wheels to PyPI by accident.

We, actually, don't differentiate these cases and want users to reproduce exactly the same results as CI service. Are you OK to keep arch = 'none' by default to keep auditwheel as default path, but for users who knows what they are doing, add arch = 'auto' ? Auto can guess both architecture and glibc (or musl) version to automatically set arch (reference https://github.com/openvinotoolkit/openvino/blob/master/src/bindings/python/wheel/CMakeLists.txt#L17-L82)

python_abi = 'none'

I've tried and it works, thank you! Do you know some hints when can users leverage these new features? BTW, I've tried to use your changes from branch:

[build-system]
requires = ["git+https://github.com/tttapa/py-build-cmake.git@rework-0.2.0", "pybind11", "pybind11-stubgen", "cmake>=3.27"]
build-backend = "py_build_cmake.build"

and it looks like such syntax is not supported by frontends unfortunately.

tttapa commented 3 weeks ago

I'll have a more detailed look at this later this week, I'll need to see how other backends such as setuptools handle platform/architecture tags.

To answer your second question, I believe that the correct syntax would be:

[build-system]
requires = [
    "py-build-cmake@git+https://github.com/tttapa/py-build-cmake@sha1-hash",
]
build-backend = "py_build_cmake.build"

Since the development branches are by definition unstable, I'd strongly recommend referencing a specific commit rather than a branch name.

Case in point, I've created a new tools.py-build-cmake.wheel section, and moved the python_abi option to it, in preparation to maybe adding options like arch to it in the future: https://github.com/tttapa/py-build-cmake/commit/dc21bfeb206279fb348a056da76bb6faf551d4be

ilya-lavrenov commented 2 weeks ago

Hi @tttapa

Maybe the better idea is to allow to call auditwheel during py-build-cmake build process? So, it will automatically fix wheel to reflect actual platform.