Fail to compile dependencies which in turn have renamed dependencies

[usbalbin]

Problem

I have two cargo projects, lets call them "the_bin" and "rename_deps".

"rename_deps" has a renamed dependency:

# rename_deps/Cargo.toml

[package]
name = "rename_deps"
version = "0.1.2"
edition = "2021"
publish = ["gitea"]

[dependencies]
embedded-hal-0-2 = { package = "embedded-hal", version = "0.2.7" }

// rename_deps/src/lib.rs
pub fn foo(_x: impl embedded_hal_0_2::PwmPin) { // <-- Using the renamed name here
    todo!()
}

This is then what gets published to my private registry

[package]
edition = "2021"
name = "rename_deps"
version = "0.1.2"
publish = ["gitea"]

[dependencies.embedded-hal-0-2]
version = "0.2.7"
package = "embedded-hal"

Compiling this works fine. I have then published this to my own private cargo registry(gitea). The publish check thing passes with no errors.

I then create a new binary project "the_bin", add "rename_deps" as dependency. And now cargo build fails to compile "rename_deps" when running cargo build on "the_bin". Same thing but with rename_deps { path = "some-path" } or git = and it compiles fine again.

   Compiling rename_deps v0.1.2 (registry `gitea`)
error[E0433]: failed to resolve: use of undeclared crate or module `embedded_hal_0_2`
 --> /home/albin/.cargo/registry/src/my-very-secret-gitea-registry/rename_deps-0.1.2/src/lib.rs:2:21
  |
2 | pub fn foo(_x: impl embedded_hal_0_2::PwmPin) {
  |                     ^^^^^^^^^^^^^^^^ use of undeclared crate or module `embedded_hal_0_2`
  |
help: there is a crate or module with a similar name
  |
2 | pub fn foo(_x: impl embedded_hal::PwmPin) {
  |                     ~~~~~~~~~~~~

For more information about this error, try `rustc --explain E0433`.
error: could not compile `rename_deps` (lib) due to 1 previous error

Steps

1. cargo new rename_deps --lib
2. Add renamed dependency
3. Use renamed dependency in rename_deps's code
4. cargo publish
5. cargo new the_bin
6. cargo add rename_deps
7. cargo build
8. Boom - Fails to build rename_deps

Possible Solution(s)

Since what I receive from my registry seems correct this is likely not due to me using a private registry I think?

Notes

This is what gets published to my private registry (ignoring the comments)

[package]
edition = "2021"
name = "rename_deps"
version = "0.1.2"
publish = ["gitea"]

[dependencies.embedded-hal-0-2]
version = "0.2.7"
package = "embedded-hal"

Version

cargo 1.79.0 (ffa9cf99a 2024-06-03)
release: 1.79.0
commit-hash: ffa9cf99a594e59032757403d4c780b46dc2c43a
commit-date: 2024-06-03
host: x86_64-unknown-linux-gnu
libgit2: 1.7.2 (sys:0.18.3 vendored)
libcurl: 8.6.0-DEV (sys:0.4.72+curl-8.6.0 vendored ssl:OpenSSL/1.1.1w)
ssl: OpenSSL 1.1.1w  11 Sep 2023
os: Manjaro 24.0.2 (Wynsdey) [64-bit]

[usbalbin]

Link to urlo thread here

[ehuss]

Can you check the index entries of your registry to verify that they are correct? I'm uncertain of the best way for you to do that, but if the registry supports the sparse protocol, it would be something like fetching https://my-registry.example.com/re/na/rename_deps.

Some registries have had issues with generating the index correctly for renamed deps. There is a section below https://doc.rust-lang.org/cargo/reference/registry-index.html#json-schema which explains some of the subtle concerns registry implementers should consider.

[usbalbin]

Can you check the index entries of your registry to verify that they are correct? I'm uncertain of the best way for you to do that, but if the registry supports the sparse protocol, it would be something like fetching https://my-registry.example.com/re/na/rename_deps.

Some registries have had issues with generating the index correctly for renamed deps. There is a section below https://doc.rust-lang.org/cargo/reference/registry-index.html#json-schema which explains some of the subtle concerns registry implementers should consider.

I assume this is incorrect? https://my-gitea-domain/api/packages/SomeOwner/cargo/re/na/rename_deps :

{"name":"rename_deps","vers":"0.1.2","deps":[{"name":"embedded-hal","req":"^0.2.7","features":[],"optional":false,"default_features":true,"target":null,"kind":"normal","registry":"https://github.com/rust-lang/crates.io-index","package":null}],"cksum":"5a4e09f3752e42de84106f0f3b3e219eb553c78ade6ba0f4473fd5b744d18572","features":{},"yanked":false}

same thing formatted:

{
    "name": "rename_deps",
    "vers": "0.1.2",
    "deps": [
        {
            "name": "embedded-hal", // <-- Should have been my new name?
            "req": "^0.2.7",
            "features": [],
            "optional": false,
            "default_features": true,
            "target": null,
            "kind": "normal",
            "registry": "https://github.com/rust-lang/crates.io-index",
            "package": null // <-- Should be "embedded-hal"?
        }
    ],
    "cksum": "5a4e09f3752e42de84106f0f3b3e219eb553c78ade6ba0f4473fd5b744d18572",
    "features": {},
    "yanked": false
}

[usbalbin]

Not sure how things work, but ~/.cargo/registry/src/my-very-secret-gitea-registry/rename_deps-0.1.2/Cargo.toml contains the correct things right?

[package]
edition = "2021"
name = "rename_deps"
version = "0.1.2"
publish = ["gitea"]

[dependencies.embedded-hal-0-2]
version = "0.2.7"
package = "embedded-hal"

[ehuss]

Yea, it looks like the registry doesn't support renamed dependencies. You might want to follow up with them.

contains the correct things right?

Yea, the manifest looks correct. However, cargo uses the data from the index for doing resolution.

[usbalbin]

[...] However, cargo uses the data from the index for doing resolution.

Oh, ok.

Yea, it looks like the registry doesn't support renamed dependencies. You might want to follow up with them.

Ok, I will give it a try.

Sorry for the noise and thanks for the help! :)

Feel free to close if you believe this issue is on the registry's side of things

[usbalbin]

I have had another gitea and/or cargo related issue. Not sure where the issue lies here either. Not sure if this is in any way related so please let me know if you prefer I open a new issue.

I have another package in the same registry(with no renamed deps). I have successfully published 0.1.5-alpha.1 which works fine. However after that I tried to upload 0.2.0-alpha.0 which also went fine, however trying to use this as a dependency does not work with cargo build not finding that version. However cargo search does find it.

Published a new version 0.2.0-alpha.1 but same thing:

$ cargo publish
...
Uploading my-crate v0.2.0-alpha.1 (/path/my-crate)     
    Uploaded my-crate v0.2.0-alpha.1 to registry `gitea`                                                                                                              
note: Waiting for `my-crate v0.2.0-alpha.1` to be available at registry `gitea`.                          
You may press ctrl-c to skip waiting; the crate should be available shortly.                                                                                                    
warning: timed out waiting for `my-crate v0.2.0-alpha.1` to be available in registry `gitea`
note: The registry may have a backlog that is delaying making the crate available. The crate should be available soon.
$ cd /path-to/my-bin
$ cargo add my-crate@0.2.0-alpha.1
    Updating `gitea` index
error: the crate `my-crate@0.2.0-alpha.1` could not be found in registry index.
$ cargo search my-crate
    my-crate = "0.2.0-alpha.1"    # 
$ cargo build
    Updating `gitea` index
    Updating crates.io index
error: failed to select a version for the requirement `my-crate = "^0.2.0-alpha.1"`
candidate versions found which didn't match: 0.1.5-alpha.1, 0.1.5-alpha.0, 0.1.4, ...
location searched: `gitea` index
required by package `my-bin v0.1.0 (/path/my-bin)`
if you are looking for the prerelease package it needs to be specified explicitly
    my-crate = { version = "0.1.5-alpha.1" }

[ehuss]

You'll need to verify that the registry's index got updated (using the similar approach above of fetching the index entry manually using something like curl). The message here:

warning: timed out waiting for my-crate v0.2.0-alpha.1 to be available in registry gitea

indicates that the index was not updated.

The reason cargo search can work is because it uses the registry API which likely goes through gitea's database. The index uses a plain HTTPS fetch. If the gitea is doing any sort of caching, then there could be a delay until the webserver's cache is cleared or evicted.

[ehuss]

I'm going to close since these seem like gitea issues. I recommend following up with them.

[usbalbin]

Here are the last working version and the first problematic version, side by side from https://my-gitea-domain/api/packages/SomeOwner/cargo/fu/nc/function-generator

[older versions here]
{"name":"function-generator","vers":"0.1.5-alpha.1","deps":[{"name":"atomic","req":"^0.5.3","features":[],"optional":false,"default_features":false,"target":null,"kind":"normal","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"const_soft_float","req":"^0.1.4","features":["no_std"],"optional":false,"default_features":true,"target":null,"kind":"normal","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"defmt","req":"^0.3","features":[],"optional":true,"default_features":true,"target":null,"kind":"normal","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"embedded-hal","req":"^0.2.7","features":["unproven"],"optional":false,"default_features":true,"target":null,"kind":"normal","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"embedded-svc","req":"^0.24.0","features":[],"optional":true,"default_features":true,"target":null,"kind":"normal","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"fixed","req":"^1.22.1","features":["num-traits","serde"],"optional":false,"default_features":true,"target":null,"kind":"normal","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"json","req":"^0.12.4","features":[],"optional":true,"default_features":true,"target":null,"kind":"normal","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"log","req":"^0.4.17","features":[],"optional":false,"default_features":true,"target":null,"kind":"normal","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"micromath","req":"^2.0.0","features":[],"optional":false,"default_features":true,"target":null,"kind":"normal","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"nb","req":"^1.0.0","features":[],"optional":false,"default_features":true,"target":null,"kind":"normal","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"num-traits","req":"^0.2.15","features":[],"optional":false,"default_features":false,"target":null,"kind":"normal","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"pid","req":"^4.0.1-alpha.0","features":[],"optional":false,"default_features":true,"target":null,"kind":"normal","registry":null,"package":null},{"name":"serde","req":"^1.0","features":["derive"],"optional":true,"default_features":false,"target":null,"kind":"normal","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"serde_json","req":"^1.0.83","features":[],"optional":true,"default_features":false,"target":null,"kind":"normal","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"stm32g0xx-hal","req":"^0.2.1-alpha.0","features":[],"optional":true,"default_features":true,"target":null,"kind":"normal","registry":null,"package":null},{"name":"stm32g4xx-hal","req":"^0.0.3-alpha.0","features":[],"optional":true,"default_features":true,"target":null,"kind":"normal","registry":null,"package":null},{"name":"anyhow","req":"^1","features":[],"optional":false,"default_features":true,"target":null,"kind":"build","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"embuild","req":"^0.31.0","features":[],"optional":true,"default_features":true,"target":null,"kind":"build","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"npm_rs","req":"^0.2.1","features":[],"optional":true,"default_features":true,"target":null,"kind":"build","registry":"https://github.com/rust-lang/crates.io-index","package":null}],"cksum":"0faa7a51521db3da51849be01e847273b05daad989518cf20e6f6f0840a722c2","features":{"default":[],"std":[],"stm32g030":["stm32g0xx-hal/stm32g030"],"stm32g474":["stm32g4xx-hal/stm32g474"]},"yanked":false}
{"name":"function-generator","vers":"0.2.0-alpha.0","deps":[{"name":"atomic","req":"^0.5.3","features":[],"optional":false,"default_features":false,"target":null,"kind":"normal","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"const_soft_float","req":"^0.1.4","features":["no_std"],"optional":false,"default_features":true,"target":null,"kind":"normal","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"defmt","req":"^0.3","features":[],"optional":true,"default_features":true,"target":null,"kind":"normal","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"embedded-hal","req":"^0.2.7","features":["unproven"],"optional":true,"default_features":true,"target":null,"kind":"normal","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"embedded-svc","req":"^0.24.0","features":[],"optional":true,"default_features":true,"target":null,"kind":"normal","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"fixed","req":"^1.22.1","features":["num-traits","serde"],"optional":false,"default_features":true,"target":null,"kind":"normal","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"json","req":"^0.12.4","features":[],"optional":true,"default_features":true,"target":null,"kind":"normal","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"log","req":"^0.4.17","features":[],"optional":false,"default_features":true,"target":null,"kind":"normal","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"micromath","req":"^2.0.0","features":[],"optional":false,"default_features":true,"target":null,"kind":"normal","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"nb","req":"^1.0.0","features":[],"optional":false,"default_features":true,"target":null,"kind":"normal","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"num-traits","req":"^0.2.15","features":[],"optional":false,"default_features":false,"target":null,"kind":"normal","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"pid","req":"^4.0.1-alpha.0","features":[],"optional":false,"default_features":true,"target":null,"kind":"normal","registry":null,"package":null},{"name":"serde","req":"^1.0","features":["derive"],"optional":true,"default_features":false,"target":null,"kind":"normal","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"serde_json","req":"^1.0.83","features":[],"optional":true,"default_features":false,"target":null,"kind":"normal","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"stm32g0xx-hal","req":"^0.2.1-alpha.0","features":[],"optional":true,"default_features":true,"target":null,"kind":"normal","registry":null,"package":null},{"name":"stm32g4xx-hal","req":"^0.0.3-alpha.0","features":[],"optional":true,"default_features":true,"target":null,"kind":"normal","registry":null,"package":null},{"name":"anyhow","req":"^1","features":[],"optional":false,"default_features":true,"target":null,"kind":"build","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"embuild","req":"^0.31.0","features":[],"optional":true,"default_features":true,"target":null,"kind":"build","registry":"https://github.com/rust-lang/crates.io-index","package":null},{"name":"npm_rs","req":"^0.2.1","features":[],"optional":true,"default_features":true,"target":null,"kind":"build","registry":"https://github.com/rust-lang/crates.io-index","package":null}],"cksum":"1deba9ce69b5f1e018ab8ef08566312e11c2365d36a2f5e1f98060c1abb41003","features":{"default":[],"e-hal-v02":["embedded-hal-0-2"],"std":[],"stm32g030":["stm32g0xx-hal/stm32g030"],"stm32g474":["stm32g4xx-hal/stm32g474"]},"yanked":false}
[newer non-working versions here]

Any ideas as to why cargo build would not find the later version?

[usbalbin]

The reason cargo search can work is because it uses the registry API which likely goes through gitea's database. The index uses a plain HTTPS fetch. If the gitea is doing any sort of caching, then there could be a delay until the webserver's cache is cleared or evicted.

How would I test this/find information about how to test this beyond the above?

[ehuss]

After publishing, cargo tries for 1 minute to verify that the package shows up in the index. The timed out waiting for... message indicates that the registry is not getting updated right away. You can also verify that by trying to download the index entry manually and see if the new version is there.

So, the method to test it would be to cargo publish, and then notice that the index isn't updated (either via cargo timing out, or manually downloading the index file and checking if it is updated).

One way this can happen is if the index is served by a webserver that does caching. If the registry does not invalidate the cache, then you will get stale entries for as long as the cache entry stays live.

There is information for registry implementers about caching and cache invalidation at https://doc.rust-lang.org/nightly/cargo/reference/registry-index.html#caching.

[usbalbin]

[..] You can also verify that by trying to download the index entry manually and see if the new version is there.

So that should be what I did in my comment above? However as far as I can tell, that does show the new versions that cargo build does not see

[usbalbin]

The result from the search api at: http://my-gitea-domain/api/packages/SomeOwner/cargo/api/v1/crates?q=function-generator

{"crates":[{"name":"function-generator","max_version":"0.2.0-alpha.2","description":""}],"meta":{"total":1}}

which matches the version of my latest attempted push which is also present in the full list from which I in my comment above only showed two lines(the index page actually showed all)

[usbalbin]

Is there anything else that could explain why cargo build only sees the first set of versions?

I have also deleted ~/.cargo/registry/ in case that does anything with caching. As far as I know there is no extra server side or proxy caching in my registry which runs on a local server with only 3 users. Still the same error:

error: failed to select a version for the requirement `function-generator = "^0.2.0-alpha.2"`
candidate versions found which didn't match: 0.1.5-alpha.1, 0.1.5-alpha.0, 0.1.4, ...