search

bazelbuild / bazel

a fast, scalable, multi-language and extensible build system
https://bazel.build
Apache License 2.0
23.2k stars 4.06k forks source link

Inefficient caching of Android binary artifacts Vs libraries/tests. #18565

Open oliviernotteghem opened 1 year ago

oliviernotteghem commented 1 year ago

Description of the feature request:

TL;DR; In order to properly populate remote cache for developers, one has to build both android binaries and android/jvm libraries. This is costly and inefficient, as it requires double CI capacity/jobs. Can we please fix this?

When building android binaries, target platform is changed to android and artifacts are not shared when, let's say, a developer build the corresponding android library(ies) to run locally a unit test. In this case, the target platform is the local machine. This means that, in order to have cache hits for both building project android binaries and project libraries/unit test/robolectric tests, we need to build both separately on CI. As these are not sharing output artifacts, jobs takes twice longer. Our CI cost essentially is doubled.

Fixing this means for most company significant $ savings. This problem is exacerbated by the fact we need to run build separately on both linux and mac.

What underlying problem are you trying to solve with this feature?

Avoiding having to rebuild project twice on CI when populating remote cache for local devs.

Which operating system are you running Bazel on?

Mac/Linux

What is the output of bazel info release?

No response

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

No response

Have you found anything relevant by searching the web?

Found an undocumented flag 'experimental_platform_in_output_dir' which could potentially help.

Any other information, logs, or outputs that you want to share?

No response

fmeum commented 1 year ago

One half of this is on the Bazel 7 roadmap, the other one is probably covered by https://github.com/bazelbuild/bazel/issues/6526. Both are actively being worked on.

oliviernotteghem commented 5 months ago

@fmeum @ahumesky : any update on this by any chance?

fmeum commented 5 months ago

Yes, this should be working with 7.2.0, I'll explain the details in a discussion some time next week.

oliviernotteghem commented 1 month ago

@fmeum : can you please provide details? Thanks!

oliviernotteghem commented 2 weeks ago

@ahumesky : bumping this issue for visibility. After our chat at BazelCon2024 you said we might be able to fix this with a transition? cc @nkoroste