search

mozilla / sccache

Sccache is a ccache-like tool. It is used as a compiler wrapper and avoids compilation when possible. Sccache has the capability to utilize caching in remote storage environments, including various cloud storage options, or alternatively, in local storage.
Apache License 2.0
5.85k stars 552 forks source link

sccache-action usage: Use GHA as local cache #2142

Open Alphare opened 7 months ago

Alphare commented 7 months ago

This is a draft PR to implement the base for a fix for https://github.com/Mozilla-Actions/sccache-action/issues/81.

Some users have been implementing their own fix by going around the sccache-action and making their own cache in GHAc (https://github.com/Mozilla-Actions/sccache-action/issues/50), but I think that this allows us to be more generic by moving more of the complexity into sccache itself, which makes it easier for end-users and allows us to re-use the same principle for other remote storage systems in the future.

This will need to be accompanied by a separate PR in sccache-action to make use of the new config option(s) and the new --update-cache command.

I've listed TODOs in the code as questions for reviewers, hence the draft state.

codecov-commenter commented 7 months ago

Codecov Report

Attention: Patch coverage is 9.32203% with 107 lines in your changes missing coverage. Please review.

Project coverage is 30.22%. Comparing base (0cc0c62) to head (464138d). Report is 71 commits behind head on main.

Files Patch % Lines
src/cache/gha.rs 0.00% 69 Missing :warning:
src/cache/cache.rs 18.91% 25 Missing and 5 partials :warning:
src/config.rs 25.00% 4 Missing and 2 partials :warning:
src/server.rs 50.00% 0 Missing and 2 partials :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #2142 +/- ## ========================================== - Coverage 30.91% 30.22% -0.69% ========================================== Files 53 53 Lines 20112 20590 +478 Branches 9755 10029 +274 ========================================== + Hits 6217 6224 +7 - Misses 7922 8281 +359 - Partials 5973 6085 +112 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Xuanwo commented 7 months ago

Fascinating idea! Actually, I believe it could be a broader solution applicable to all storage services. For instance, downloading from S3/GCS/AZBlob should also be useful.

What do you think?

Alphare commented 7 months ago

Fascinating idea! Actually, I believe it could be a broader solution applicable to all storage services. For instance, downloading from S3/GCS/AZBlob should also be useful.

What do you think?

Indeed, I even mentioned this briefly in this PR's first message. I think this can make sense, but I didn't want to start with a more generic implementation before I could get a sense of how correct my solution is to this problem.

Is there any chance you could answer the few questions I've left as comments in the code?

Alphare commented 6 months ago

I think this is ready for a first review. I'll try to somehow get sccache-action to test this in the meantime.

sylvestre commented 6 months ago

I guess you saw that some jobs are failing

Alphare commented 6 months ago

After a long battle I finally managed in a (very very) hacky way to exercise this path¹.

I'll have to make adjustments here when debugging.

[1] https://github.com/Mozilla-Actions/sccache-action/actions/runs/8803242248/job/24160865310

Alphare commented 6 months ago

I have failed to fully get this to a working state because of an unknown error (in the gha-as-local integration test), and because I've now truly ran out of time on this, but I feel like this is very close to working for a single version.

I have detailed my thoughts about the remaining work in a TODO inside the code to make it easier to contextualize for a further effort, as agreed to with @sylvestre in a private chat.