Currently, if SageMath is built, it will use up more than number of parallel jobs specified by MAKEFLAGS, because make and ninja has no communication.
I notice that there are already tools to parse number of threads (./src/bin/sage-num-threads.py). At the very least we can throw -jN to ninja, so it would use at most 2N jobs.
For reference, at the moment (I don't fully understand how it work) pyproject_hooks is used to invoke build_wheel, which in turn delegates to mesonpy.
def build_wheel(wheel_directory, config_settings, metadata_directory=None):
"""Invoke the mandatory build_wheel hook.
If a wheel was already built in the
prepare_metadata_for_build_wheel fallback, this
will copy it rather than rebuilding the wheel.
"""
prebuilt_whl = _find_already_built_wheel(metadata_directory)
if prebuilt_whl:
shutil.copy2(prebuilt_whl, wheel_directory)
return os.path.basename(prebuilt_whl)
return _build_backend().build_wheel(
wheel_directory, config_settings, metadata_directory
)
We need to modify config_settings, which are done by modifying hook_input in
If I understood the code correctly, it suffices to set config_settings={"compile-args": ["-jN"]} (for some integer N), which is done by setting hook_input to {"kwargs": {"config_settings": {"compile_args": ["-jN"]}}}.
I don't know how hook_input is controlled.
Looks like it comes from sdh_build_wheel which calls python3 -m build, and you can somehow pass --config-setting=--compile-args=-jN to it using $#build 1.2.2.post1
which is in turn managed by sage/build/pkgs/cython/spkg-install.in
In top level pyproject.toml there is build-backend = 'mesonpy' specification. So passing -jN to that backend should be safe?
In sage-env, it already sets SAGE_NUM_THREADS and SAGE_NUM_THREADS_PARALLEL environment variables.
Currently, if SageMath is built, it will use up more than number of parallel jobs specified by
MAKEFLAGS, becausemakeandninjahas no communication.I notice that there are already tools to parse number of threads (
./src/bin/sage-num-threads.py). At the very least we can throw-jNto ninja, so it would use at most2Njobs.Partial work:
A process tree might look like the following.
For reference, at the moment (I don't fully understand how it work)
pyproject_hooksis used to invokebuild_wheel, which in turn delegates tomesonpy.We need to modify
config_settings, which are done by modifyinghook_inputinIf I understood the code correctly, it suffices to set
config_settings={"compile-args": ["-jN"]}(for some integerN), which is done by settinghook_inputto{"kwargs": {"config_settings": {"compile_args": ["-jN"]}}}.I don't know how
hook_inputis controlled.Looks like it comes from
sdh_build_wheelwhich callspython3 -m build, and you can somehow pass--config-setting=--compile-args=-jNto it using$#build 1.2.2.post1which is in turn managed by
sage/build/pkgs/cython/spkg-install.inIn top level
pyproject.tomlthere isbuild-backend = 'mesonpy'specification. So passing-jNto that backend should be safe?In
sage-env, it already setsSAGE_NUM_THREADSandSAGE_NUM_THREADS_PARALLELenvironment variables.