LukeMathWalker / cargo-chef

A cargo-subcommand to speed up Rust Docker builds using Docker layer caching.
Apache License 2.0
1.72k stars 113 forks source link

Unintended Full Recompilation When Integrating Cargo Chef into Graph Node Dockerfile #267

Closed paymog closed 5 months ago

paymog commented 5 months ago

I'm trying to add cargo chef to this open source project. I've modified the Dockerfile like so. When I try to build this, I see a full recompilation happening during the main build command. I also see a warning emitted during the cooking phase

Step 18/53 : RUN RUSTFLAGS="-g" cargo chef cook --release --recipe-path recipe.json
 ---> Running in 7b1bb385cb88
warning: virtual workspace defaulting to `resolver = "1"` despite one or more workspace members being on edition 2021 which implies `resolver = "2"`
note: to keep the current resolver, specify `workspace.resolver = "1"` in the workspace root's manifest
note: to use the edition 2021 resolver, specify `workspace.resolver = "2"` in the workspace root's manifest
note: for more details see https://doc.rust-lang.org/cargo/reference/resolver.html#resolver-versions

Here's the recipe.json file that gets generated.

The docker build command I use is

docker build --target graph-node \
            --build-arg "COMMIT_SHA=$(git rev-parse --short HEAD)" \
            --build-arg "REPO_NAME=$(git remote get-url origin | cut -d ":" -f 2 | cut -d "." -f 1)" \
            --build-arg "BRANCH_NAME=$(git branch --show-current)" \
            --build-arg "TAG_NAME=$(git describe --tags --abbrev=0)" \
            -t graph-node:$(git branch --show-current) \
            -f docker/Dockerfile --progress=plain .

I checked one of the *.d files under target/release/deps and I see the hardcoded paths starting with /graph-node which seems correct to me.

LukeMathWalker commented 5 months ago

On the surface, it looks correct. Do you have build logs to share?

LukeMathWalker commented 5 months ago

(The resolver warning has nothing to do with cargo-chef, that's a cargo warning that you need to handle)

paymog commented 5 months ago

Yup, here are some build logs: https://gist.github.com/paymog/b71cc9d87add2f8087f97f70ad257f4c

FWIW I don't see that resolver warning when building without cargo-chef

LukeMathWalker commented 5 months ago

Yup, here are some build logs: https://gist.github.com/paymog/b71cc9d87add2f8087f97f70ad257f4c

FWIW I don't see that resolver warning when building without cargo-chef

These are the build logs from a fresh run (i.e. this line would come from the cache otherwise). I need the build logs for a second run that has a warm cache to understand if things are working or not.

paymog commented 5 months ago

Whoops, sorry about this. These logs should have that stage cached.

LukeMathWalker commented 5 months ago

From the logs I can infer that you have a rust-toolchain.toml file and that it points to an older version of Rust than the one bundled in cargo-chef's Docker image. The best course of action, in that case, is to "sync" the Rust version ahead of time—see this as an example.

paymog commented 5 months ago

Hmm, I'm not using the cargo-chef docker image - I'm using the chef planner image as shown here

LukeMathWalker commented 5 months ago

It's the same, you're getting the latest Rust toolchain but your toolchain file wants 1.68.

paymog commented 5 months ago

That seems to have fixed it, thank you for your help!