base64 is missing when vendoring Rust package/crates for offline build

[johanneskastl]

Hi,

openSUSE packager here. I just tried to update to 0.22.0, but get an error, as the base64 dependency seems to be missing from the vendoring in our offline builds:

Could you have a look, please?

Thanks and kind Regards, Johannes

[   66s] + RUSTFLAGS=' -Clink-arg=-Wl,-z,relro,-z,now -C debuginfo=2 -C incremental=false -C strip=none'
[   66s] + /usr/bin/cargo auditable build -j4 --offline --release --features docgen,rdcore
[   66s] warning: `default_features` is deprecated in favor of `default-features` and will not work in the 2024 edition
[   66s] (in the `nix` dependency)
[   66s] error: no matching package named `base64` found
[   66s] location searched: registry `crates-io`
[   66s] required by package `coreos-installer v0.22.0 (/home/abuild/rpmbuild/BUILD/coreos-installer-0.22.0)`
[   66s] As a reminder, you're using offline mode (--offline) which can sometimes cause surprising resolution failures, if this error is too confusing you may wish to retry without the offline flag.
[   66s] error: Bad exit status from /var/tmp/rpm-tmp.RawEun (%build)

[prestist]

@johanneskastl yeah absolutely will take a look

[prestist]

@johanneskastl so I just looked into the vendor'd file i.e I downloaded it put it in 0.22.0, and ran

cargo test --all-features --config 'source.crates-io.replace-with="vv"' --config 'source.vv.directory="vendor"'

I am not seeing a missing dependency. Picking at straws have you tried re-downloading and or verifying sha?

[johanneskastl]

Thanks for checking, @prestist!

Hmmm, I just checked the vendor tarball created before entering the offline build environment. I indeed see two directories in vendor, one called base64 and one called base64-0.21.7.

So, with my limited Rust expertise I found that the Cargo.lock contains two different versions of the package:

[[package]]
name = "base64"
version = "0.21.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9d297deb1925b89f2ccc13d7635fa0714f12c87adce1c75356b39ca9b7178567"

[[package]]
name = "base64"
version = "0.22.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6"

Could it be that the second one, 0.22.1 (which is a dependency for reqwest, rustls-pemfile and others if I understand the file correctly) is missing? Or is not recognized because the directory is just called base64?

Kind Regards, Johannes

[johanneskastl]

Maybe @firstyear could take a look whether this has been caused by the OBS cargo_vendor service?

[prestist]

@johanneskast no worries! and I am sorry you are going through this.

Could it be that the second one, 0.22.1 (which is a dependency for reqwest, rustls-pemfile and others if I understand the file correctly) is missing? Or is not recognized because the directory is just called base64?

The first version will be the crate name, regardless of version, i.e, base64. If there are other versions they will be placed in a folder with the version number. You can read about it here => https://doc.rust-lang.org/cargo/commands/cargo-vendor.html under the --versioned-dirs argument which we do not use so we have default functionality. i.e Normally versions are only added to disambiguate multiple versions of the same package.

After downloading tar -xzf coreos-installer-0.22.0-vendor.tar.gz cargo build --offline --all-features --config 'source.crates-io.replace-with="vv"' --config 'source.vv.directory="vendor"' I am still getting a successful build. I am unsure of what could cause this. Notably I am the packager of it, so its possible there is some magic dust causing it to work on this computer.

@yasminvalim could you try to build this offline using the uploaded vendor for 0.22.0.

[prestist]

Also, I am working on getting a 0.22.1 release out (soon). It has a few package updates and will be re-vendor'd and such. It might be good to try and see if that works for you. (https://github.com/coreos/coreos-installer/issues/1495)

[johanneskastl]

In case of the openSUSE package, the vendoring is done by the cargo_vendor service that uses the files in the git checkout of the corresponding version. But let's see if 0.22.1 makes this error go away... :-)

[prestist]

Well, 0.22.1 was just released upstream; @johanneskastl; We also did have a base64 dependabot pr land between these two releases. Im not sure if its coincidence but its noteable.

[johanneskastl]

OK, I found the underlying issue. This was caused by a packaging mistake I made. Sorry for the noise and thanks for your help!

[prestist]

No worries, glad you found it!