mozilla / sccache

Sccache is a ccache-like tool. It is used as a compiler wrapper and avoids compilation when possible. Sccache has the capability to utilize caching in remote storage environments, including various cloud storage options, or alternatively, in local storage.
Apache License 2.0
5.85k stars 552 forks source link

Replace rouille with warp #2241

Open Saruniks opened 3 months ago

Saruniks commented 3 months ago

Part of porting changes from cachepot https://github.com/mozilla/sccache/issues/1620, https://github.com/paritytech/cachepot/pull/131

Xuanwo commented 3 months ago

Wow, great work! Thanks in advance; Please ping me while you think this PR is ready.

By the way, could you add @montekki as a co-author on this PR? I think it would be great to give them recognition.

codecov-commenter commented 3 months ago

Codecov Report

Attention: Patch coverage is 7.54717% with 49 lines in your changes missing coverage. Please review.

Project coverage is 41.09%. Comparing base (0cc0c62) to head (134e1a2). Report is 71 commits behind head on main.

Files Patch % Lines
src/dist/http.rs 0.00% 26 Missing :warning:
tests/system.rs 23.52% 13 Missing :warning:
src/dist/client_auth.rs 0.00% 8 Missing :warning:
src/util.rs 0.00% 2 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #2241 +/- ## =========================================== + Coverage 30.91% 41.09% +10.18% =========================================== Files 53 55 +2 Lines 20112 20857 +745 Branches 9755 9844 +89 =========================================== + Hits 6217 8571 +2354 - Misses 7922 8176 +254 + Partials 5973 4110 -1863 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Saruniks commented 3 months ago

I'll add montekki as a co-author.

montekki, please let us know if you have any comments on this PR or if you would like to takeover the work.


I'm not sure how to prepare this for a merge. Fixing the merge conflicts and getting the CI to pass was straightforward, but there are a lot of changes, and there's a risk for fallout. I'll give an update after testing this on my setup.


I wonder what's up with the freebsd tests though.

Saruniks commented 3 months ago

I did some testing. For some reason sccache_dist scheduler doesn't work properly without SCCACHE_NO_DAEMON=1.

I tested and it even works like this: SCCACHE_NO_DAEMON=1 sccache-dist scheduler --config /etc/sccache/scheduler.conf &

This is the same reason why freebsd ci doesn't pass. I'll try to fix this issue.

I don't have any useful logs yet.

Saruniks commented 3 months ago

I found out that I need to create a tokio::Runtime instance manually after call to daemonize, instead o using the #[tokio::main] attribute.

Xuanwo commented 3 months ago

All unit test for freebsd passed, we only have a No distributed compilations succeeded. Seems the dist-server or dist-client doesn't start correctly on freebsd.

We expect to see something like:

Statistics of second buildtest
  Compile requests                      13
  Compile requests executed              4
  Cache hits                             4
  Cache hits (Rust)                      4
  Cache misses                           0
  Cache hits rate                   100.00 %
  Cache hits rate (Rust)            100.00 %
  Cache timeouts                         0
  Cache read errors                      0
  Forced recaches                        0
  Cache write errors                     0
  Compilation failures                   0
  Cache errors                           0
  Non-cacheable compilations             0
  Non-cacheable calls                    8
  Non-compilation calls                  1
  Unsupported compiler calls             0
  Average cache write                0.000 s
  Average compiler                   0.000 s
  Average cache read hit             0.000 s
  Failed distributed compilations        0
Saruniks commented 3 months ago

I fixed freebsd test by creating tokio::Runtime manually instead of using #[tokio::main]:

    let runtime = Runtime::new().context("Failed to create Tokio runtime")?;
    runtime.block_on(async { http_scheduler.start().await })?;

It was a problem that I noticed while testing on ubuntu too.

Xuanwo commented 3 months ago

cc @sylvestre would you like to take another look?