LukeMathWalker / cargo-chef

A cargo-subcommand to speed up Rust Docker builds using Docker layer caching.
Apache License 2.0
1.73k stars 115 forks source link

Docker build error: candidate versions found which didn't match: 0.0.1 #93

Closed orhun closed 7 months ago

orhun commented 3 years ago

Hello, thanks for this crate. I'm using it for the Docker builds of my git-cliff project for a couple of months now. But recently I got an error in my CI workflow:

 Step 8/20 : RUN cargo chef cook --release --recipe-path recipe.json
 ---> Running in e89886c52ec8
    Updating crates.io index
error: failed to select a version for the requirement `git-cliff-core = "^0.2.5"`
candidate versions found which didn't match: 0.0.1
location searched: /app/git-cliff-core
required by package `git-cliff v0.0.1 (/app/git-cliff)`
thread 'main' panicked at 'Exited with status code: 101', /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/cargo-chef-0.1.30/src/recipe.rs:145:27
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
The command '/bin/sh -c cargo chef cook --release --recipe-path recipe.json' returned a non-zero code: 101

Dockerfile: https://github.com/orhun/git-cliff/blob/72cee9caa632498c55a7c5f1aac2657fbb445b90/Dockerfile

I have already seen #84 and although it seems like the same issue, I realized I can build it just fine with cargo-chef v0.1.23 so I suspect this is happening because of the things changed between v0.1.23-v0.1.24.

I'm not sure how cargo-chef works internally so I cannot debug this further for now. Any ideas about how to fix this?

LukeMathWalker commented 3 years ago

Thanks for reporting! It should be fixed in 0.1.31.

orhun commented 3 years ago

It's building without errors now. Thanks!

sebastiendan commented 2 years ago

Hi, I'm getting the same error now with v0.1.41. You can reproduce it by preparing the recipe with the following repo (all local deps get 0.0.1 version): https://github.com/novifinancial/winterfell

Emilgardis commented 2 years ago

I've managed to make a minimal repro of this (or, a similar issue)

https://github.com/Emilgardis/cargo-chef-issue93

docker build .
Sending build context to Docker daemon  4.322MB
Step 1/17 : FROM lukemathwalker/cargo-chef:latest AS chef
 ---> cb6ae6b94ee0
Step 2/17 : RUN rustup default nightly
 ---> Using cache
 ---> 1894e37b5a40
Step 3/17 : RUN curl -L https://github.com/stedolan/jq/releases/download/jq-1.6/jq-linux64 > /usr/bin/jq; chmod +x /usr/bin/jq
 ---> Using cache
 ---> 1b0616eb6605
Step 4/17 : WORKDIR /app
 ---> Using cache
 ---> da44d27d0f5f
Step 5/17 : FROM chef AS planner
 ---> da44d27d0f5f
Step 6/17 : ARG CARGO_UNSTABLE_SPARSE_REGISTRY=true
 ---> Using cache
 ---> 0fd6d29a81d8
Step 7/17 : COPY . .
 ---> f4055e60998c
Step 8/17 : RUN cargo chef prepare --recipe-path recipe.json
 ---> Running in 7088bab4414a
Removing intermediate container 7088bab4414a
 ---> 785c079352e1
Step 9/17 : RUN jq -C '.' recipe.json
 ---> Running in 2998f8a3569f
{
  "skeleton": {
    "manifests": [
      {
        "relative_path": "Cargo.toml",
        "contents": "bin = []\nbench = []\ntest = []\nexample = []\n\n[package]\nname = \"chef-test\"\nedition = \"2021\"\nversion = \"0.0.1\"\nauthors = []\nkeywords = []\ncategories = []\nautobins = true\nautoexamples = true\nautotests = true\nautobenches = true\npublish = true\n[dependencies.libc]\nversion = \"0.0.1\"\n\n[lib]\npath = \"src/lib.rs\"\nname = \"chef_test\"\ntest = true\ndoctest = true\nbench = true\ndoc = true\nplugin = false\nproc-macro = false\nharness = true\nedition = \"2021\"\nrequired-features = []\ncrate-type = [\"rlib\"]\n"
      },
      {
        "relative_path": "libc/Cargo.toml",
        "contents": "bin = []\nbench = []\ntest = []\nexample = []\n\n[package]\nname = \"libc\"\nedition = \"2021\"\nversion = \"0.0.1\"\nauthors = []\nkeywords = []\ncategories = []\nautobins = true\nautoexamples = true\nautotests = true\nautobenches = true\npublish = true\n\n[dependencies]\n\n[lib]\npath = \"src/lib.rs\"\nname = \"libc\"\ntest = true\ndoctest = true\nbench = true\ndoc = true\nplugin = false\nproc-macro = false\nharness = true\nedition = \"2021\"\nrequired-features = []\ncrate-type = [\"rlib\"]\n"
      }
    ],
    "config_file": null,
    "lock_file": "version = 3\n\n[[package]]\nname = \"chef-test\"\nversion = \"0.0.1\"\ndependencies = [\"libc\"]\n\n[[package]]\nname = \"libc\"\nversion = \"0.0.1\"\nsource = \"registry+https://github.com/rust-lang/crates.io-index\"\nchecksum = \"8371e4e5341c3a96db127eb2465ac681ced4c433e01dd0e938adbef26ba93ba5\"\n"
  }
}
Removing intermediate container 2998f8a3569f
 ---> 811238d9ad38
Step 10/17 : FROM chef AS builder
 ---> da44d27d0f5f
Step 11/17 : ARG CARGO_UNSTABLE_SPARSE_REGISTRY=true
 ---> Using cache
 ---> 0fd6d29a81d8
Step 12/17 : COPY --from=planner /app/recipe.json recipe.json
 ---> 18d5e5115bcd
Step 13/17 : RUN cargo chef cook --recipe-path recipe.json
 ---> Running in 20832604201e
    Updating crates.io index
error: failed to select a version for the requirement `libc = "^0.0.1"` (locked to 0.0.1)
candidate versions found which didn't match: 0.2.132, 0.2.131, 0.2.129, ...
location searched: crates.io index (which is replacing registry `crates-io`)
required by package `chef-test v0.0.1 (/app)`
perhaps a crate was updated and forgotten to be re-vendored?
thread 'main' panicked at 'Exited with status code: 101', /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/cargo-chef-0.1.41/src/recipe.rs:161:27
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
The command '/bin/sh -c cargo chef cook --recipe-path recipe.json' returned a non-zero code: 101

changing https://github.com/Emilgardis/cargo-chef-issue93/blob/18adb31dc5726a00f8aa2a53cf8f9ea002b03336/Cargo.toml#L9-L10 makes it work as expected

libc = "0.2.135"
docker build .
Sending build context to Docker daemon  4.322MB
Step 1/17 : FROM lukemathwalker/cargo-chef:latest AS chef
 ---> cb6ae6b94ee0
Step 2/17 : RUN rustup default nightly
 ---> Using cache
 ---> 1894e37b5a40
Step 3/17 : RUN curl -L https://github.com/stedolan/jq/releases/download/jq-1.6/jq-linux64 > /usr/bin/jq; chmod +x /usr/bin/jq
 ---> Using cache
 ---> 1b0616eb6605
Step 4/17 : WORKDIR /app
 ---> Using cache
 ---> da44d27d0f5f
Step 5/17 : FROM chef AS planner
 ---> da44d27d0f5f
Step 6/17 : ARG CARGO_UNSTABLE_SPARSE_REGISTRY=true
 ---> Using cache
 ---> 0fd6d29a81d8
Step 7/17 : COPY . .
 ---> 1624747528e6
Step 8/17 : RUN cargo chef prepare --recipe-path recipe.json
 ---> Running in 5be63d302957
Removing intermediate container 5be63d302957
 ---> 3e148099d6ac
Step 9/17 : RUN jq -C '.' recipe.json
 ---> Running in 73ce326142c8
{
  "skeleton": {
    "manifests": [
      {
        "relative_path": "Cargo.toml",
        "contents": "bin = []\nbench = []\ntest = []\nexample = []\n\n[package]\nname = \"chef-test\"\nedition = \"2021\"\nversion = \"0.0.1\"\nauthors = []\nkeywords = []\ncategories = []\nautobins = true\nautoexamples = true\nautotests = true\nautobenches = true\npublish = true\n\n[dependencies]\nlibc = \"0.2.132\"\n\n[lib]\npath = \"src/lib.rs\"\nname = \"chef_test\"\ntest = true\ndoctest = true\nbench = true\ndoc = true\nplugin = false\nproc-macro = false\nharness = true\nedition = \"2021\"\nrequired-features = []\ncrate-type = [\"rlib\"]\n"
      },
      {
        "relative_path": "libc/Cargo.toml",
        "contents": "bin = []\nbench = []\ntest = []\nexample = []\n\n[package]\nname = \"libc\"\nedition = \"2021\"\nversion = \"0.0.1\"\nauthors = []\nkeywords = []\ncategories = []\nautobins = true\nautoexamples = true\nautotests = true\nautobenches = true\npublish = true\n\n[dependencies]\n\n[lib]\npath = \"src/lib.rs\"\nname = \"libc\"\ntest = true\ndoctest = true\nbench = true\ndoc = true\nplugin = false\nproc-macro = false\nharness = true\nedition = \"2021\"\nrequired-features = []\ncrate-type = [\"rlib\"]\n"
      }
    ],
    "config_file": null,
    "lock_file": "version = 3\n\n[[package]]\nname = \"chef-test\"\nversion = \"0.0.1\"\ndependencies = [\"libc\"]\n\n[[package]]\nname = \"libc\"\nversion = \"0.0.1\"\nsource = \"registry+https://github.com/rust-lang/crates.io-index\"\nchecksum = \"8371e4e5341c3a96db127eb2465ac681ced4c433e01dd0e938adbef26ba93ba5\"\n"
  }
}
Removing intermediate container 73ce326142c8
 ---> 4736c9f8a74d
Step 10/17 : FROM chef AS builder
 ---> da44d27d0f5f
Step 11/17 : ARG CARGO_UNSTABLE_SPARSE_REGISTRY=true
 ---> Using cache
 ---> 0fd6d29a81d8
Step 12/17 : COPY --from=planner /app/recipe.json recipe.json
 ---> Using cache
 ---> 6cd273423e2e
Step 13/17 : RUN cargo chef cook --recipe-path recipe.json
 ---> Using cache
 ---> 2ef0c9e789cc
Step 14/17 : COPY . .
 ---> b8f139b29db3
Step 15/17 : RUN cargo build
 ---> Running in 2ae80fcec972
   Compiling libc v0.2.132
   Compiling chef-test v0.1.0 (/app)
    Finished dev [unoptimized + debuginfo] target(s) in 0.79s
Removing intermediate container 2ae80fcec972
 ---> 67c9135bbdf3
Step 16/17 : FROM debian:buster-slim AS runtime
 ---> 6d8fb53a1e62
Step 17/17 : WORKDIR /app
 ---> Using cache
 ---> a9c49a1d6d47
Successfully built a9c49a1d6d47
sushshring commented 1 year ago

This same issue occurs if a package has a git source. For example, using the plaid repo for rust adds the following to Cargo.toml: plaid = { git = "https://github.com/telcoin/plaid.git", version = "0.9.1"}. This makes cargo-chef set the version to 0.0.1 in the recipe.json. I think @sullivan-sean's fix could be extended to this case as well.

LukeMathWalker commented 1 year ago

@sullivan-sean's reproduction leaves me a bit puzzled: it's basically a case where you have a local crate with the same name of a crate that exists on crates.io, but you have no path dependency to the local one and only depend on the one stored in the public registry. When/why would that be useful? 🤔

treamology commented 1 year ago

I'm facing a similar issue when trying to use cargo-chef with LibAFL. The libafl package has a dependency on libafl_derive which is in the same workspace, specified with a path dependency:

libafl_derive = { version = "0.10.0", optional = true, path = "../libafl_derive" }

Running cargo chef cook generates the following error:

#0 1.836 error: failed to select a version for the requirement `libafl_derive = "^0.0.1"`
#0 1.836 candidate versions found which didn't match: 0.10.0
#0 1.836 location searched: /libafl_src/libafl_derive
#0 1.836 required by package `libafl v0.10.0 (/libafl_src/libafl)`
LukeMathWalker commented 1 year ago

If you can provide steps for me to reproduce I can try to have a look @treamology.

Emilgardis commented 1 year ago

This is most likely the same error as in my repro

https://github.com/AFLplusplus/LibAFL/blob/721c02cd2c4a439db0acd5f19d853dd7ec118bbb/libafl/Cargo.toml#L59

th0masb commented 1 year ago

I was seeing this error after updating some of my deps, it looked something like:

 > [builder 2/4] RUN cargo chef cook --release --recipe-path recipe.json:
#13 0.592     Updating crates.io index
#13 91.56 error: failed to select a version for the requirement `aho-corasick = "^1.0.0"`
#13 91.56 candidate versions found which didn't match: 0.7.20, 0.7.19, 0.7.18, ...
#13 91.56 location searched: crates.io index
#13 91.56 required by package `regex v1.8.1`
#13 91.56     ... which satisfies dependency `regex = "^1.8.1"` (locked to 1.8.1) of package `myopic-board v0.0.1 (/build/engine/board)`
#13 91.56 thread 'main' panicked at 'Exited with status code: 101', /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/cargo-chef-0.1.33/src/recipe.rs:145:27
#13 91.56 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

where it was using cargo-chef-0.1.33. Updating the cargo-chef image fixed it but the error message produced here was confusing.

andresilva91 commented 1 year ago

Is there any updates on this? I'm getting the same error on 0.1.61. Is there a version I can install that won't have this issue? I'll try 0.1.31 for now, but it's 2 years old Edit: 0.1.31. also has this issue for me

Alibirb commented 1 year ago

@sullivan-sean's reproduction leaves me a bit puzzled: it's basically a case where you have a local crate with the same name of a crate that exists on crates.io, but you have no path dependency to the local one and only depend on the one stored in the public registry. When/why would that be useful?

I have a use case for that setup. I'm developing a library locally, but I want to build and test my application with the library version on Github before I push commits in the application, to ensure the build will work for everyone else, so I temporarily change the Cargo.toml to point to Github instead of the local folder.

LukeMathWalker commented 7 months ago

This should have been solved by #247.