search

mozilla / sccache

Sccache is a ccache-like tool. It is used as a compiler wrapper and avoids compilation when possible. Sccache has the capability to utilize caching in remote storage environments, including various cloud storage options, or alternatively, in local storage.
Apache License 2.0
5.85k stars 552 forks source link

Intermittent "error: Connection to server timed out", "error: Could not compile `style`" when building mozilla-central locally #399

Open theres-waldo opened 5 years ago

theres-waldo commented 5 years ago

(Originally filed in BMO).

I build mozilla-central locally with sccache configured via the following line in my .mozconfig:

mk_add_options 'export RUSTC_WRAPPER=sccache'

My build often fails with the following error:

 1:22.87 error: Connection to server timed out
 1:22.92 error: Could not compile `style`.

It's usually resolved just by running the build again, but still somewhat annoying.

theres-waldo commented 5 years ago

(To be clear, this is just a plain local build, with sccache used to cache Rust compilations to reduce incremental build time when switching branches and such. I'm not trying to do any distributed compilation here.)

jwatt commented 5 years ago

(To be clear, this is just a plain local build, with sccache used to cache Rust compilations to reduce incremental build time [snip]

For Mozilla code I think rust incremental compilation is on for all configurations (I could well be wrong) and I guess that's the case for most other projects too. It looks like sccache currently doesn't cache rust compiler output if rustc incremental compilation is turned on in which case for most people sccache isn't going to help with compiling Rust code unless you're distributing the compilation.

theres-waldo commented 5 years ago

Interesting; I wasn't aware of that. So I should just remove that line from my .mozconfig for the time being?

jwatt commented 5 years ago

I'm guessing so from my current understanding (at least until #236 is fixed), but maybe better to solicit the opinion of someone who knows more about these things than me. Or else just run some test builds with and without export RUSTC_WRAPPER=sccache to get concrete numbers for your specific setup.

luser commented 5 years ago

cargo will only use incremental compilation for path-based dependencies, so all the crates that live in mozilla-central will use it. It will not use incremental compilation for all of the dependencies from crates.io, so sccache should be able to cache those compiles.

jwatt commented 5 years ago

Thanks for clarifying, @luser!

jwatt commented 5 years ago

I just did a few tests for my setup for building the Mozilla source.

Roughly speaking a build using icecc and ccache (populated cache) takes about 20 minutes to do a clobber build. If I also use RUSTC_WRAPPER=sccache (populated ccache) it takes about 15-16 minutes.

So RUSTC_WRAPPER=sccache can indeed provide a significant build time improvement, at least for certain scenarios. I guess it depends on how often the crates from crates.io that we build change from commit to commit. Then again, RUSTC_WRAPPER=sccache didn't seem to add much overhead in the case that the sccache cache was empty, so it seems worth having regardless.