d-e-s-o
commented
3 months ago Interestingly this timeout is also hit when using direct file system access as opposed to HTTP. So that would rule out any issues related to caching.
Open StackOverflowExcept1on opened 3 months ago
d-e-s-o
commented
3 months ago Interestingly this timeout is also hit when using direct file system access as opposed to HTTP. So that would rule out any issues related to caching.
d-e-s-o
commented
3 months ago I think the issue comes down to cargo failing to validate some entry:
0.732203346s INFO main:exec: cargo::sources::registry::index: failed to parse "de/mo/demo-a" registry package: optional dependency `renamed_once_cell` is not included in any feature
Make sure that `dep:renamed_once_cell` is included in one of features in the [features] table.It ends up discarding it more or less silently, which means it won't show up in the summaries that it queries every second https://github.com/rust-lang/cargo/blob/ffa9cf99a594e59032757403d4c780b46dc2c43a/src/cargo/ops/registry/publish.rs#L258
StackOverflowExcept1on
commented
3 months ago I fixed this issue by removing the unused dependency and feature in our code, but when publish the package to crates-io I didn't get this issue. This still says that cargo-http-registry is quite unstable.
d-e-s-o
commented
3 months ago If you can provide crate name and version that you published to crates.io as well as the contents of the local registry after you tried publishing the same versioned crate to it, we may be able to get to the bottom of it.
StackOverflowExcept1on
commented
3 months ago I tried to publish the crate gcore, which has the feature dep:codec (which is actually a renamed dependency of parity-scale-codec). The contents of the crates.io registry: https://github.com/rust-lang/crates.io-index/blob/master/gc/or/gcore, which is slightly different from the local registry. In particular, it does not have features2.
crates.io:
{
"name": "gcore",
"vers": "1.5.0",
"deps": [
{
"name": "codec",
"req": "^3.6.4",
"features": [],
"optional": true,
"default_features": false,
"target": null,
"kind": "normal",
"package": "parity-scale-codec"
},
{
"name": "galloc",
"req": "^1.5.0",
"features": [],
"optional": false,
"default_features": true,
"target": null,
"kind": "dev"
},
{
"name": "gear-core-errors",
"req": "^1.5.0",
"features": [],
"optional": false,
"default_features": true,
"target": null,
"kind": "normal"
},
{
"name": "gear-stack-buffer",
"req": "^1.5.0",
"features": [],
"optional": false,
"default_features": true,
"target": null,
"kind": "normal"
},
{
"name": "gprimitives",
"req": "^1.5.0",
"features": [],
"optional": false,
"default_features": false,
"target": null,
"kind": "normal"
},
{
"name": "gsys",
"req": "^1.5.0",
"features": [],
"optional": false,
"default_features": true,
"target": null,
"kind": "normal"
},
{
"name": "hex-literal",
"req": "^0.4.1",
"features": [],
"optional": false,
"default_features": true,
"target": null,
"kind": "dev"
}
],
"cksum": "c0bdb11c2fe9c5e92615a7ea4b188408bfbe557c47809c7adeefc018efdf707d",
"features": {
"debug": []
},
"features2": {
"codec": [
"dep:codec",
"gear-core-errors/codec",
"gprimitives/codec"
]
},
"yanked": false,
"rust_version": "1.80",
"v": 2
}cat /tmp/cargo-http-registry/gc/or/gcore | jq
{
"name": "gcore",
"vers": "1.5.0-b710fb843",
"deps": [
{
"name": "codec",
"req": "^3.6.4",
"features": [],
"optional": true,
"default_features": false,
"target": null,
"kind": "normal",
"registry": "https://github.com/rust-lang/crates.io-index",
"package": "parity-scale-codec"
},
{
"name": "gear-core-errors",
"req": "^1.5.0-b710fb843",
"features": [],
"optional": false,
"default_features": true,
"target": null,
"kind": "normal",
"registry": null,
"package": null
},
{
"name": "gear-stack-buffer",
"req": "^1.5.0-b710fb843",
"features": [],
"optional": false,
"default_features": true,
"target": null,
"kind": "normal",
"registry": null,
"package": null
},
{
"name": "gprimitives",
"req": "^1.5.0-b710fb843",
"features": [],
"optional": false,
"default_features": false,
"target": null,
"kind": "normal",
"registry": null,
"package": null
},
{
"name": "gsys",
"req": "^1.5.0-b710fb843",
"features": [],
"optional": false,
"default_features": true,
"target": null,
"kind": "normal",
"registry": null,
"package": null
},
{
"name": "galloc",
"req": "^1.5.0-b710fb843",
"features": [],
"optional": false,
"default_features": true,
"target": null,
"kind": "dev",
"registry": null,
"package": null
},
{
"name": "hex-literal",
"req": "^0.4.1",
"features": [],
"optional": false,
"default_features": true,
"target": null,
"kind": "dev",
"registry": "https://github.com/rust-lang/crates.io-index",
"package": null
}
],
"cksum": "ed514b238616c25adaa68fe08846241444b10492200e2e42c860456058af8e88",
"features": {
"codec": [
"gear-core-errors/codec",
"gprimitives/codec"
],
"debug": [],
"ethexe": [
"gsys/ethexe"
]
},
"yanked": false,
"links": null
}I also described a minimal example here: https://github.com/rust-lang/cargo/issues/14399#issuecomment-2291332275, but I was told that the absence of features2 is normal
StackOverflowExcept1on
commented
3 months ago I don't see much difference with crates-io. Some fields are in different order, some fields are explicitly set to null (we can try to change the order and add #[serde(skip_serializing_if = "Option::is_none")]). The ethexe feature was added to our unreleased build, but not in version 1.5.0, so don't pay attention to it. Only difference that really makes sense is the absence of dep:codec and features2 in cargo-http-registry. Also on crates-io there seems to be some sorting by certain dependency fields.
I also tried to replicate crates-io behavior locally, but I couldn't get dep:codec from package publish data. Features were split into features and features2, but dep:codec was still missing. It's possible that crates-io parses Cargo.toml or something like that.
https://github.com/rust-lang/crates.io/blob/9a026aa8838c858a5ce5c4389331007e8561152c/src/models/krate.rs#L504
d-e-s-o
commented
3 months ago Ah okay, thanks for the analysis. Very strange. Yeah, I don't think ordering matters. Same for null. I may try debug further locally at some point, if Cargo folks don't respond.
The registry does not respond within 1 minute to request from cargo. My cargo config looks like this: