search

keith / rules_multirun

Bazel rules for running multiple commands in parallel in a single bazel invocation
Apache License 2.0
79 stars 16 forks source link

feat: Support concurrency limit #55

Open honnix opened 5 months ago

honnix commented 5 months ago

This is a PoC about how jobs can be used to control concurrency limit.

Currently jobs can either be 0 (no limit, spawn/fork as many as possible), or 1 (serial). This PR uses a thread pool to control the max number of concurrently running processes.

This PR also solves a deadlock when a child process outputs too much data that don't fit into output buffer.

honnix commented 5 months ago

@keith Could you please take a look at this and let me know what you think? Thanks.

honnix commented 2 weeks ago

@keith Does this change make sense? We have been using a patched version for a while and it seems to work well.