search

ilbers / isar

Integration System for Automated Root filesystem generation
Other
183 stars 73 forks source link

Support for buildchroot pooling for parallel builds #85

Open Majlen opened 2 years ago

Majlen commented 2 years ago

I'm wondering, if there are any plans to support chroot pooling to enable building several targets in parallel?

Currently each build needs to acquire a lock against a single buildchroot first. This essentially means that setting BB_NUMBER_THREADS is somewhat meaningless (yes, at least fetches run in parallel, ...), since there is always only 1 build running at each moment. This can be fairly limitting when building image from highly modular layers (i.e. building many small packages), as each of these packages takes about 50s on my machine just to install the build dependencies (and these steps also acquire the dpkg lock). Adding a new package to the image means adding another >1m step into the build (even though it would be possible to build it in parallel, as many steps are not really bottlenecked on any resource).

An option would be to create a pool of N buildchroots (which can be fairly cheap on modern filesystems with reflinks), where N could be taken from BB_NUMBER_TREADS. Not sure if it can be done, as I'm not 100% familiar with ISAR / BitBake internals, but sounds doable?

jan-kiszka commented 2 years ago

Builds are currently read-lock protected against modifications of the buildchroot because dpkg can get unhappy otherwise (don't have the change at hand but it's in the git history). So, you can have multiple builds running in parallel, but once one is done and wants to deploy its deb files, that is serialized against other builds, thus may delay the build of packages that depend on it. Same for installing build dependencies vs. building some other packages in parallel.

Most of this will be resolved by the upcoming sbuild changes which will privatize the buildchroot. Will come soon after the currently pending release (likely a matter of weeks only).

Majlen commented 2 years ago

OK, that would be definitely better than the situation I currently see in 0.8.

It is not perfect, but if only the deploys are serialized, then at least the packages can be built in parallel, once they have their dependencies ready...