Rust and incremental compilation

[alexcrichton]

With the advent of incremental compilation on stable Rust compile times will hopefully get much better especially in debug mode and hopefully eventually in release mode with https://github.com/rust-lang/rust/issues/48996. Ideally sccache would help to integrate with incremental compilation to even further accelerate CI and builds whenever possible!

Unfortunately it's not super clear how sccache would integrate with incremental compilation in Rust. There will never be an exact cache hit as sccache already otherwise does that, so we sort of need an "almost cache hit" to load something from the cache. One possible idea may be to hash fewer things and upload full compiler incremental cache directories. That way sccache would download the previous cache directory and let rustc take care of what should be reset.

In any case this isn't super pressing for now as incremental for release builds is still aways out, but figured I'd open an issue!

[luser]

I had asked about this a while back when I was doing the Rust work initially but I think incremental compilation was still new enough that I couldn't get a useful answer. It would definitely be interesting to figure out a smart plan here. Maybe the simplest thing we could do would be to store the incremental compilation outputs in the cache entry, so that if you get a full cache hit you can at least do an incremental compile on top of that if you make changes?

You and @michaelwoerister and I should find time at the SF All Hands in June to chat about this.

[alexcrichton]

Maybe the simplest thing we could do would be to store the incremental compilation outputs in the cache entry, so that if you get a full cache hit you can at least do an incremental compile on top of that if you make changes?

For sure! That I think sounds like a great idea and is quite possible to do with -C incremental=.. I think. (in that sccache would recognize such an argument, parse it, and slurp it up)

Unfortunately though I won't actually be at the SF All Hands, but @michaelwoerister if you've got a spare moment in the near future I'd love to chat with you about your thoughts on this :)

[michaelwoerister]

Maybe the simplest thing we could do would be to store the incremental compilation outputs in the cache entry, so that if you get a full cache hit you can at least do an incremental compile on top of that if you make changes?

Yes, that should work. It will at least double the amount of data to download though. We'd also need to review if there's anything non-portable in the cache files (i.e. local file paths or something).

[luser]

Yes, that should work. It will at least double the amount of data to download though. We'd also need to review if there's anything non-portable in the cache files (i.e. local file paths or something).

sccache doesn't currently get cache hits if file paths change (https://github.com/mozilla/sccache/issues/196), so this isn't any worse than the current situation.

[jakelee8]

@Xuanwo What do you think about supporting Rust incremental builds or disabling incremental compilation (by default or only for crate libraries)?

Caveat docs:

1. https://github.com/mozilla/sccache#rust
2. https://github.com/mozilla/sccache/blob/main/docs/Rust.md

[WieeRd]

I'm a little confused about this caveat, does this mean I need to disable incremental build completely and globally via build.incremental option in ~/.cargo/config.toml if I want to benefit from cached builds? My primary reason for using sccache is to share 3rd party dependency build caches between projects on my desktop.

[Tudyx]

I'm kind of confused too. Since https://github.com/mozilla/sccache/pull/259/ sccache seems not to cache rustc command that contains -C incremental. But I don't know if, in that case, it will still cache other artifacts. In other words I don't know if there is any value to have both incremental and sccache enabled? Maybe for fresh build?

[luser]

But I don't know if, in that case, it will still cache other artifacts.

I'm not sure exactly what you're asking here. If the compiler invocation contains -C incremental, then sccache will not cache that compile. Most Rust projects invoke the compiler quite a few times to compile all their dependencies. Last I knew incremental compilation was enabled by default only for crates that are members of the workspace, but not for dependencies.

In any event, the primary use case for which sccache was designed—and at which it excels—is compiling in CI, where you should not be using incremental compilation at all.

[Tudyx]

Thanks for taking the time to respond.

Last I knew incremental compilation was enabled by default only for crates that are members of the workspace, but not for dependencies.

By analyzing the output of cargo -vv , I confirmed that's still true.

I'm not sure exactly what you're asking here.

Indeed it was not clear. What I meant is that other calls to rustc, that don't contain -C incremental, will still be cached by scache. So, for instance, they will still have some rlib stored in sccache for dependencies that don't belong to the worskpace, even if you use incremental compilation. They will also be in the target folder.

My point here is that if you suppress your target folder and then do fresh build, you will still have better performance if you had scache + incremental enabled because it will be able to use the previously cached rlib right ?

Correct me if I'm wrong, but it seems then to have value to use both sccache and incremental compilation locally. Of course that's not something users should rely on because, as you rightfully pointed out, it diverges from sccache main use case, which is CI.