search

canonical / charmcraft

Collaborate, build and publish charmed operators for Kubernetes, Linux and Windows.
Apache License 2.0
64 stars 66 forks source link

No support for paralell builds for multi-base charms #861

Open fnordahl opened 1 year ago

fnordahl commented 1 year ago

For a charm with multiple bases it would be useful to be able to optionally have charmcraft build those bases in parallel. With charmcraft's use of build providers such as LXD this should in theory be possible to implement.

The reasoning behind this request comes from the need to build charms from source and/or for non-x86/arm architectures. When building from source, most of the build time is spent in pip's dependency resolution algorithm, and very little time is spent actually compiling code. Other architectures currently don't have pre-compiled wheels shipped on PyPi, so everyone would need to build from source for those architectures.

The lack of support for this natively in charmcraft could be worked around by interpreting the charmcraft.yaml and running multiple invocations of charmcraft with an appropriate --bases-index argument.

But it feels like an awful lot of work to delegate to the end user, and it would be much simpler with a --base-build-concurrency N argument or something similar?

facundobatista commented 1 year ago

We could have a charm-base-build-concurrency option in the Charm plugin for this. Default to 1, so no behaviour change at all if you don't incorporate the option. Of course, it can not be set to >1 if using "destructive mode".

Beyond having to prefix all log lines generated during the different builds so they can be tracked differently when reading the logs, we had the problem on what will happen with the terminal output during this parallel builds (as there will be two or more processes outputting to the same terminal). Probably we should enforce "un-managed output" in Craft CLI when Chramcraft packs in parallel (the same behaviour that when the output is redirected to a file).