search

mdavidsaver / setuptools_dso

setuptools extension for building non-Python Dynamic Shared Objects
Other
9 stars 6 forks source link

Limit parallel compile on hosts with limited memory #26

Open mdavidsaver opened 1 year ago

mdavidsaver commented 1 year ago

https://github.com/mdavidsaver/setuptools_dso/blob/0b7223087d0d25b43fc31f095d0987513002a5e3/src/setuptools_dso/dsocmd.py#L354

It would be helpful to detect hosts with many cores, but limited RAM. (eg. a Pi with 4 cores but only 1GB of RAM) Such systems often can't compile multiple objects without paging onto very slow storage, which defeats the purpose of parallel compilation.

I see two parts:

mdavidsaver commented 1 year ago

f82966c2e78e8fe0ff16edb6cd87e9d22e864a1d attempts to apply a (probably crude) test looking for 512MB of available RAM per prospective parallel job. This is only done when run on Linux.