We're going to enable new ARM64 runners in CI, where test-run is invoked in a Docker container. At the same time, the runner is an LXD container created by the service provider. In this circumstances, the Docker container sees all the 128 online CPUs, but the runner may have only some of them available (depending on the pricing plan).
Python 3.3+ has a function to determine available CPUs, so we can reduce the parallelism to this value. We fall back to the online CPUs count on Python < 3.3.
After this change, test-run follows a CPU affinity mask set by taskset (and, I guess, by numactl).
The change is similar to replacing nproc --all to nproc.
The change only affects the default behavior, which can be overwritten by passing the --jobs (or -j) CLI option or using the TEST_RUN_JOBS environment variable.
coverage: 62.607% (+0.002%) from 62.605%
when pulling f33c3243abddd0be7e37241d72b090a8206ff8d4 on Totktonada/calc-jobs-using-available-cpus
into 84ebae5d041125ab3bb52514c1472db844c45f20 on master.
coverage: 62.607% (+0.002%) from 62.605%
when pulling bec97bf2d593039134c41f6b4045ac39ac4393aa on Totktonada/calc-jobs-using-available-cpus
into 84ebae5d041125ab3bb52514c1472db844c45f20 on master.
coverage: 62.607% (+0.002%) from 62.605%
when pulling 4f926ba7fe1e4f173991393bf6bc7e88069d6f29 on Totktonada/calc-jobs-using-available-cpus
into 84ebae5d041125ab3bb52514c1472db844c45f20 on master.
coverage: 62.675% (+0.07%) from 62.605%
when pulling 66bef07baa78f918d629debbb244a8fb82734513 on Totktonada/calc-jobs-using-available-cpus
into 84ebae5d041125ab3bb52514c1472db844c45f20 on master.
We're going to enable new ARM64 runners in CI, where test-run is invoked in a Docker container. At the same time, the runner is an LXD container created by the service provider. In this circumstances, the Docker container sees all the 128 online CPUs, but the runner may have only some of them available (depending on the pricing plan).
Python 3.3+ has a function to determine available CPUs, so we can reduce the parallelism to this value. We fall back to the online CPUs count on Python < 3.3.
After this change, test-run follows a CPU affinity mask set by
taskset
(and, I guess, bynumactl
).The change is similar to replacing
nproc --all
tonproc
.The change only affects the default behavior, which can be overwritten by passing the
--jobs
(or-j
) CLI option or using theTEST_RUN_JOBS
environment variable.Reported in https://github.com/tarantool/tarantool/pull/10102