Closed nagisa closed 1 month ago
Our current CI tests a combination of all crates in the workspace, which can lead to issues related to crate features especially which are an additive combination. In particular a different crate may be enabling a feature crucial for a crate, meaning the faulty crate would not compile in isolation.
Something like what's in https://github.com/near/nearcore/pull/11983
I'm running into the following problem: We want to dry-run
publish crates near-*
. Assume this includes:
near-bar
which has no dependency on other packages from near-*
near-foo
which depends on on near-bar
cargo ws publish ...
publishes crates in an order that allows satisfying dependencies. Though the point of dry-run
is to not actually publish the crates to a registry. Accordingly cargo publish --dry-run -p near-bar
succeeds but cargo publish --dry-run -p near-foo
fails because near-bar
at the new version to be dry-run
published cannot be found in the registry.
What if near-bar
is a path dependency? My understanding of cargo docs is that this doesn't matter for packaging, which will ignore the path:
Cargo will ignore the path key for dependencies in published packages.
There seems to be no option to enforce using the path dependency. Intuitively this makes sense, as all dependencies of a published crate should be published too.
I can think of the following workarounds:
--dry-run
to NEAR's cargo-workspaces
fork and try to implement a workaround there. I doubt this would be practicable as under the hood cargo
commands are invoked.Perhaps I'm missing another workaround that is more promising?
Dummy registry would probably be ideal approach here as it would emulate the actual workload we're looking to test here the best. There are some projects that provide such a thing, https://github.com/hirevo/alexandrie/ seems most straightforward from a brief look?
List of all sorts of tools are here: https://github.com/rust-lang/cargo/wiki/Third-party-registries.
Thanks for the pointers. I'll give the dummy registry approach a try.
Some further issues popped up after setting up a dummy registry and publishing crates to them:
cargo ws publish --registry ...
can publish a crate to the dummy registry, but then it times out because it waits for the crate to become available on crates.io. Even though it was made aware of the alternate registry with --registry
. Further investigations are needed to find out if this is an issue with cargo-workspaces
upstream or the fork used in nearcore
.
kellnr
) is sending an incorrect response after the crate was uploaded to it, triggering the unexpected behavior of cargo-workspaces
cargo publish
would be possible only after figuring out the correct order in which the near-*
crates need to be published since they can depend on each other. cargo workspaces publish
can do that, but it has the issue described above.Making all of this work in CI might require going down some rabbit holes. I'm unassigning myself as I should move on to another project instead.
For anyone who might tackle this in the future, I can recommend kellnr
for setting up the dummy registry. It seemed to be the option that's easiest to setup in CI while still providing the required functionality. Heads up that while margo
is even easier to set up, it does not (yet) provide the functionality required to cargo publish
crates to it. Most software to set up alternate registries seems under active development, so in the future other options might become better suited.
Our current CI tests a combination of all crates in the workspace, which can lead to issues related to crate features especially which are an additive combination. In particular a different crate may be enabling a feature crucial for a crate, meaning the faulty crate would not compile in isolation.
Something like what's in https://github.com/near/nearcore/pull/11983