Bazel uses exactly 25% more resources (as defined via resource_set) than given with --local_cpu_resources

[lunch-glide-pepper]

Description of the bug:

I have two actions that each declare CPU usage of 1000 by passing resource_set to actions.run_shell. bazel build --local_cpu_resources=1600 ... runs both actions in parallel. bazel build --local_cpu_resources=1599 ... runs them one after the other.

I would expect the cutoff to be between 1999 and 2000 instead.

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

#!/bin/bash
set -euo pipefail

cat << 'EOF' > BUILD
load(":defs.bzl", "custom_bash_rule")

custom_bash_rule(
    name = "file1",
    command = "sleep 5; touch $OUTPUT",
    output = "file1",
)

custom_bash_rule(
    name = "file2",
    command = "sleep 5; touch $OUTPUT",
    output = "file2",
)
EOF

cat << 'EOF' > defs.bzl
# buildifier: disable=module-docstring
def _resource_set_callback(os_name, num_inputs):
    return {"cpu": 1000}

def _custom_bash_rule_impl(ctx):
    output_file = ctx.actions.declare_file(ctx.attr.output)

    ctx.actions.run_shell(
        command = "\n".join([
            "set -euo pipefail",
            'OUTPUT="' + output_file.path + '"',
            ctx.attr.command,
        ]),
        outputs = [output_file],
        resource_set = _resource_set_callback,
    )

    return [DefaultInfo(files = depset(direct = [output_file]))]

custom_bash_rule = rule(
    implementation = _custom_bash_rule_impl,
    attrs = {
        "command": attr.string(mandatory = True),
        "output": attr.string(mandatory = True),
    },
)
EOF

cat << 'EOF' > WORKSPACE
EOF

bazel clean --expunge
echo
echo "Building with --local_cpu_resources=1600"
bazel build --local_cpu_resources=1600 ...

bazel clean --expunge
echo
echo "Building with --local_cpu_resources=1599"
bazel build --local_cpu_resources=1599 ...

Output:

INFO: Starting clean (this may take a while). Consider using --async if the clean takes more than several minutes.

Building with --local_cpu_resources=1600
Starting local Bazel server and connecting to it...
INFO: Analyzed 2 targets (4 packages loaded, 7 targets configured).
INFO: Found 2 targets...
INFO: Elapsed time: 7.840s, Critical Path: 5.07s
INFO: 3 processes: 1 internal, 2 linux-sandbox.
INFO: Build completed successfully, 3 total actions
INFO: Starting clean (this may take a while). Consider using --async if the clean takes more than several minutes.

Building with --local_cpu_resources=1599
Starting local Bazel server and connecting to it...
INFO: Analyzed 2 targets (4 packages loaded, 7 targets configured).
INFO: Found 2 targets...
INFO: Elapsed time: 12.873s, Critical Path: 10.08s
INFO: 3 processes: 1 internal, 2 linux-sandbox.
INFO: Build completed successfully, 3 total actions

Which operating system are you running Bazel on?

Ubuntu 20.04

What is the output of bazel info release?

release 6.0.0

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

[benjaminp]

https://github.com/bazelbuild/bazel/blob/114d7720ea52fbf613bc3e06dd746eb07ce5c335/src/main/java/com/google/devtools/build/lib/actions/ResourceManager.java#L63-L67 https://github.com/bazelbuild/bazel/blob/114d7720ea52fbf613bc3e06dd746eb07ce5c335/src/main/java/com/google/devtools/build/lib/actions/ResourceManager.java#L151-L158

[lunch-glide-pepper]

Thanks!

Would a PR adding a mention of this to https://bazel.build/reference/command-line-reference#flag--local_cpu_resources be welcome?

Also, would it make sense to move the constants to command line flags?

[wilwell]

Yeah, @benjaminp is absolutely right.

@lunch-glide-pepper PR with flag description fix is welcome!

[github-actions[bot]]

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 30 days. It will be closed in the next 7 days unless any other activity occurs or one of the following labels is added: "not stale", "awaiting-bazeler".

[brentleyjones]

Not stale.