search

pypa / build

A simple, correct Python build frontend
https://build.pypa.io
MIT License
697 stars 115 forks source link

Add `--jobs` option to `build` #762

Closed YoSTEALTH closed 3 months ago

YoSTEALTH commented 3 months ago

Add --jobs option to build to use multiple threads. Currently compiling takes a long time in single thread. For example python3 setup.py build_ext -j16 --inplace already exists.

henryiii commented 3 months ago

Build has no control over the backend other than passing through config settings to it. Check with your backend. Scikit-build-core supports pipx run build -Ccmake.define.CMAKE_BUILD_PARALLEL_LEVEL=8 for example (and defaults to parallel builds).

YoSTEALTH commented 3 months ago

Currently I am trying to test build for https://github.com/YoSTEALTH/Liburing/blob/master/setup.py#L61 I do add how many thread to use. Running it as python3 -m build

henryiii commented 3 months ago

Setuptools only supports compiling extensions in parallel, not individual files. You have to hack it in with something like https://github.com/pybind/pybind11/blob/7f2214bc680b7cc9f6ad138a077b3d6d172b51f7/pybind11/setup_helpers.py#L367-L500 - or you can use a modern build system like scikit-build-core or meson-python.

YoSTEALTH commented 3 months ago

Looks like python3 -m build -C--global-option="build_ext -j16" solves the problem, mentioned in https://github.com/pypa/build/issues/328

It would be nice if build could expose or take advantage of such features. Maybe add build config in pyproject.toml or something.

henryiii commented 3 months ago

This is up to the backend. Scikit-build-core does let you set this in pyproject.toml. Setuptools could add it.