kraken-build / kraken

The Kraken build system.
https://kraken-build.github.io/kraken/
28 stars 12 forks source link

[FR] Support parallel job execution #194

Open NiklasRosenstein opened 10 months ago

NiklasRosenstein commented 10 months ago

Kraken currently executes only a single job at a time. Builds could most likely be sped up significantly by executing multiple jobs at the same time. We already model a DAG, and deriving which tasks can be executed in parallel based on that is trivial.

One thing we should keep in mind is formatters: When formatter tasks are not put into an explicit order by dependencies, they might already cause issues (e.g. black should run before isort). This may be aggravated by having formatters run in parallel.

Ideally the Kraken terminal output would show all currently running task names. When a task is complete, it immediately prints the output. Some tasks could require the terminal or should be run in foreground, for which we could introduce a flag on the Task class.