search

bazelbuild / bazel

a fast, scalable, multi-language and extensible build system
https://bazel.build
Apache License 2.0
23.06k stars 4.04k forks source link

Make `--experimental_remote_cache_async` not experimental and on by default #21578

Closed brentleyjones closed 3 weeks ago

brentleyjones commented 7 months ago

Description of the feature request:

It would be nice if the --experimental_remote_cache_async flag was no longer experimental and on by default. If that's not possible now, then issues should be identified and fixed as well in order to allow the flip.

Which category does this issue belong to?

Remote Execution

What underlying problem are you trying to solve with this feature?

Better build times when using a remote cache.

Which operating system are you running Bazel on?

No response

What is the output of bazel info release?

No response

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse HEAD ?

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

tjgq commented 7 months ago

I'd like this to happen at some point before the Bazel 8 release, but I'm not treating it as a priority at the moment.

tjgq commented 7 months ago

(Moving it back to untriaged so we explicitly discuss it at the next team meeting.)

meisterT commented 6 months ago

related: https://github.com/bazelbuild/bazel/issues/19273

coeuvre commented 6 months ago

I am aware of some open issues for this flag, we need to address them before make it default. Like @tjgq said, it's not a priority at the moment, but we should make this happen before Bazel 8.

tjgq commented 3 months ago

A known issue is that async uploading requires spawn outputs to not be moved/deleted until they've been uploaded. One situation where this isn't upheld is https://github.com/bazelbuild/bazel/issues/22501. Test attempts are another (each attempt produces the test outputs at their expected paths, but then Bazel moves them into a per-attempt path before running the next attempt).

tjgq commented 1 month ago

As @fmeum found out in https://github.com/bazelbuild/bazel/pull/23307, Java reduced classpath actions are another situation where files can disappear after spawn execution (causing problems for async uploading).

My current plan for 8.0.0 is to default to async uploading except for actions that move/delete their outputs. In the longer term I'd like to find a more definitive solution that lets us remove the sync code path entirely.

tjgq commented 1 month ago

Reopening since we had to roll back the flag flip in b3cc928b4b1bdff0182a9a9a31d409fd6515a2af. I expect to roll forward again in time for Bazel 8.