allow overriding dependency major version

[aep]

currently there is no way in replace or patch to specify a different version than the original dependency.

with replace:

[replace]
"openssl:0.9.24" = {git = "https://github.com/sfackler/rust-openssl.git"

error: no matching package for override `https://github.com/rust-lang/crates.io-index#openssl:0.9.24` found
location searched: https://github.com/sfackler/rust-openssl.git
version required: = 0.9.24

with patch:

[patch.crates-io]
openssl  = {git = "https://github.com/sfackler/rust-openssl.git"}

is simply ignored.

error: failed to run custom build command for `openssl v0.9.24`

With the complexity of packages growing, it is sometimes nessesary to override versions in Cargo.toml since upstream cant update the version yet if another dependency hasnt updated.

[matklad]

You probably need to update your lockfile file for patch to work:

https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#testing-a-bugfix

Next up we need to ensure that our lock file is updated to use this new version of uuid so our project uses the locally checked out copy instead of one from crates.io. The way [patch] works is that it'll load the dependency at ../path/to/uuid and then whenever crates.io is queried for versions of uuid it'll also return the local version.

This means that the version number of the local checkout is significant and will affect whether the patch is used. Our manifest declared uuid = "1.0" which means we'll only resolve to >= 1.0.0, < 2.0.0, and Cargo's greedy resolution algorithm also means that we'll resolve to the maximum version within that range. Typically this doesn't matter as the version of the git repository will already be greater or match the maximum version published on crates.io, but it's important to keep this in mind!

Note thar git repository contains openssl 0.10, while you seem to replace 0.9. That means that you probably need to update Cargo.toml as well, because 0.10 and 0.9 are not semver compatible.

[aep]

thanks, i'm aware of this. the lockfile is deleted on every try.

that is correct, i was asking for a feature that allows overriding 0.9 with 0.10 in Cargo.toml of my project rather than in the actual project that depends on 0.9. intentionally ignoring semver compatibility

[matklad]

i was asking for a feature that allows overriding 0.9 with 0.10 in Cargo.toml of my project rather than in the actual project that depends on 0.9

Oh, sorry, I've misunderstood your intentions! Yes, I think currently it is impossible to replace dependency specifications: that is, if a crate depends on foo 1.0.0, you can't make it to depend on foo 2.0.0 instead (this is a non-trivial task, because 1.0 and 2.0 are semver-incompatible).

However, it is possible to override the dependency itself. My understanding is that your project depends on library foo, which depends on openssl 0.9, and you want foo to use openssl 0.10. Would it be possible to fork foo, modify it's Cargo.toml to say openssl 0.10, and then patch foo instead of openssl?

[aep]

yep, doing that now, this helps me :) but i was thinking there should be a more generic solution for this in cargo

[matklad]

Yeah, such extension is possible in theory!

However, the current "fork upstream and edit Cargo.toml" has two benefits:

• it already works :)
• it nudges you towards submitting a PR to the upstream, which is arguably better than just accumulating project-local replacements.

[ryankurte]

i'd really like to see this too, imo it is a huge usability issue for both patch and replace, and it seems to me that without the ability to override version specs neither of them actually work for all but the simplest scenario (as is described in the docs) :-/

The fork-and-update approach is nice for the case when you're just a version behind, but, practically there are a bunch of situations that are not that and in my experience you often end up having to fork a pile of transitive dependencies to edit their dependency versions to test a patch, at which point patch and replace don't actually offer any utility and you may as well just change the dependency key directly.

As a simple example, for packages A, B, C and D, where:

• A depends on B and C
• B depends on D:0.1.0
• C depends on D:0.2.0

To test an unpublished local change to D (or if D publishes a 0.3.0 release), there's no way to patch up to that from A without forking and altering both B and C. This problem is in no way limited to only two intermediate dependencies, and as one of the maintainers of a reasonably widely used D-like package, this makes testing patches and changes a huge effort.

The documentation 1 2 3 also only mentions versions are important in one of the three places, and does not cover transitive dependencies at all, so it'd be neat to clear that up with whatever the outcome of this is.

[netvl]

Any chance of getting this implemented? Overriding dependencies manually, even if there is a potential semver violation, is an important task in many complex projects. Unfortunately, the real world is not ideal, and it is possible to have situations where dependency versions in upstream projects just don't match, and there is a need to set things manually.

Right now, I can't do anything to fix a dependency issue in my project without manually forking some repositories and doing fixes there, and then using the [patch] section in my manifest.

It does not seem at the first glance to be a hard thing to do from the technical standpoint: just allow specifying dependency version overrides in the manifest, e.g. like this:

[[version-overrides.some-crate]]
version = "1.2.3"  # override dependencies only for some-crate=1.2.3 but not e.g. some-crate=2.3.1

[version-overrides.some-crate.dependencies]
dependency = "3.2.1"  # overridden version

This configuration would work as a simple replace in the manifest of the overridden crate, and won't do anything else. So in the example above, Cargo would behave as if the manifest of some-crate=1.2.3 contained dependency = "3.2.1" instead of whatever dependency is defined there by default.

This does go against the idea of semantic versions, but as I said, sometimes there are things which have to be done, and it is great if tools give the developer a power to do them.

[ccope]

Also, it's not currently possible to patch multiple versions of the same dependency in your project. Example dependency tree from the project-level Cargo.toml:

A = 2.0
B = 1.0
    A = 2.0
    C = 1.0
        A = 1.0

A has a bug that you need to patch. You can only specify a single version because the patch is unversioned:

[patch.crates-io]
A = { git = "https://github.com/my/repo.git", branch = "fix-2.0" }

As long as C is the only user of A@0.1 you can just patch it (or as previously requested, override the constraint) to use the same version, but this doesn't scale if the dependency is widely used (which is somewhat likely due to rust's small stdlib).

[matklad]

Isn't this exactly this case from the doc?

https://doc.rust-lang.org/cargo/reference/manifest.html#using-patch-with-multiple-versions

On Fri, 21 Feb 2020 at 23:54, Cam Cope notifications@github.com wrote:

Also, it's not currently possible to patch multiple versions of the same dependency in your project. Example dependency tree from the project-level Cargo.toml:

A = 2.0 B = 1.0 A = 2.0 C = 1.0 A = 1.0

A has a bug that you need to patch. You can only specify a single version because the patch is unversioned:

[patch.crates-io] A = { git = "https://github.com/my/repo.git", branch = "fix-0.2" }

As long as C is the only user of A@0.1 you can just patch it (or as previously requested, override the constraint) to use the same version, but this doesn't scale if the dependency is widely used (which is somewhat likely due to rust's small stdlib).

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/rust-lang/cargo/issues/5640?email_source=notifications&email_token=AANB3M6OS4TLXQVUS3WDYETREBLRFA5CNFSM4FFVLY5KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMULOZA#issuecomment-589870948, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANB3M22JZBMWTNCDLIZDFLREBLRFANCNFSM4FFVLY5A .

[ccope]

Ah yes, I missed the note underneath about the name being ignored. That format is kind of confusing though, I was thinking about adding a config variant like:

package_name = [
    { version = "1.0", git = "..." },
    { version = "2.0", git = "..." },
]

Is there a reason it wasn't done that way originally?

[glandium]

Here's a real world example where this feature would be useful: sccache depends on rouille, which hasn't been updated since 2019. rouille depends on tiny_http 0.6. This setup has a bug that surfaces in sccache, and that was fixed in tiny_http 0.7. Considering the lack of activity on the rouille repo, I don't see a bump of its Cargo.toml to use tiny_http 0.7 happening. It seems the only workable solution today is to either move away from rouille (lot of work), or publish a rouille fork to crates.io. (I tried publishing a fork of tiny_http and to use a patch with a package pointing to the fork, but that doesn't seem to work)

[Eh2406]

If you are a maintained project and your dependency has not been updated, then a fork of the dependency seams like the honest way to do things.

[skhameneh]

Local fork patches appear to be ignored if the major does not match the original request.

[jflatow]

I agree this seems like a huge problem. Forking every indirect dependency in light of vulnerabilities and the like is totally unscalable / impractical for a downstream project. I would love the option to force a patch.

[fzyzcjy]

Any updates? This is a very useful feature!

[kevincox]

A problem I'm having is that I am trying out a new release of a popular library to be an early-adopter (actix-web 4) which is mostly compatible with version 3. However some libraries are now unusable because they specify version 3 in their manifests so all of the types are (confusingly) wrong. In this case there is a 99% change that the libraries work, I am ok to deal with that risk as a beta tester and it doesn't make sense to update the library (because they are following the stable version as they should!). Right now there isn't really a good option. We could maintain a branch/fork with a modified manifest but that seems pretty pointless if no code changes are required. In this case I think it makes sense to just have the ability to override the declared dependency to update it from v3 to v4 so that I can test out the new version.

[seimonw]

A problem I'm having is trying to build an old rootfs using yocto release sumo, when building cargo it goes off and gets the latest crates but eg the latest v1 bitflags (1.3.2) won't build with the sumo version of rust (1.24.1) :/

[wbrickner]

Bump, this is a really important practical problem that cargo should allow you to solve the easy way.

[OneSadCookie]

This has bitten me multiple times now, and I still don't have a practical solution to the problem.

In my current case, I'm trying to use genpdf which requires image 0.23.12, with barcoders which requires image 0.22. I'm sure I can find an image version compatible with both, but I can't see any way to force them to use a common version.

[silvestrst-crypto]

+1 this is a frequent headache for me as well.

[gustav3d]

Indeed, this is a major issue for me too.

[blueforesticarus]

I hit this problem on a weekly basis.

[overhacked]

If you are a maintained project and your dependency has not been updated, then a fork of the dependency seams like the honest way to do things.

While re-reading this issue and the comments, because this still comes up frequently for me, I haven't seen a particular argument in favor of making versions patch/replace-able in Cargo.toml: source control. Forking dependencies requires maintaining either:

1) a separate repository 2) a git submodule 3) a vendored directory in the project repo

Having a completely separate fork repo or a submodule completely divorces the dependency override from the core repository and adds a third, hidden dependency: keeping two version histories in sync, the project's head and the fork. Vendoring the fork does keep the override adjacent to the core repository, but actually adds a fourth hidden dependency: keeping the vendored code in sync with the upstream or external, forked repo via git subtree or the like.

The option proposed in this issue makes documenting and maintaining the override of a transitive dependency much simpler: two lines in Cargo.toml, a patch/replace line and a comment documenting why. No change to repo structure.

[awused]

Patch seems not to work for my project at all in all but the very simplest cases, because I'm transitively depending on a few different versions of the glib/gio/gobject already. As soon as I try to specify a local patch for gtk4 it seems like my transitive dependencies on glib get out of sync, so trait bounds stop resolving properly. Two transitive dependencies on glib 0.19.3 turn into one dependency on 0.19.3 and one on 0.19.0, which leads to many trait bound <Something>: gtk4::prelude::ObjectType is not satisfied build errors. I think I'd need a way to force all glib:0.19 compatible versions to at least use the same 0.19.x.

It ends up being far less work to just comment out large portions of my own code to eliminate those dependencies while testing gtk4-rs patches than to figure out what combination of patch lines I need. That's if any combination will even work; after reading this bug I'm pretty doubtful I could actually make it work.

[kornelski]

This feature may become much more important if Rust gets LTS releases. LTS users may want to prevent all their dependencies from requiring higher, incompatible versions of crates.

For example, a dependency like env_logger can make many crates unusable on an older Rust version, even though changes between "major" versions of the crate aren't that major, and it could easily be force-downgraded.

This is currently doable by individually patching every user of the crate to replace, but that is too laborious. In the users forum this is a recurring question, and the status quo answer is always disappointing.

[weihanglo]

@kornelski I am experimenting on patching with patch files https://github.com/rust-lang/cargo/pull/13779. This should solve the problem in the other way, since it is able to patch everything including package.version in Cargo.toml. See this test case as an example:

https://github.com/rust-lang/cargo/blob/faba4abc89f61dcb3a5141f534b07539d1cf80fd/tests/testsuite/patch_files.rs#L556-L612

(That said, I'd like to explore more on ergonomics for it)

[epage]

For example, a dependency like env_logger can make many crates unusable on an older Rust version, even though changes between "major" versions of the crate aren't that major, and it could easily be force-downgraded.

While I understand the general thought (imo another good example is dealing with packages that depend on git2), I'm curious with this example as, generally, env_logger should only be depended on by applications and not libraries, unless its for tests for which it shouldn't matter.

[kornelski]

ok, env_logger wasn't the best example. But it happens that crates have mostly-compatible semver-major versions, and with a little luck could be upgraded or downgraded: git2, gix, bindgen, itertools, hashbrown, comrak, and about 3 out of 4 cargo releases. Recently old ahash had incompatibility with nightly, and it'd be handy to be able to easily force-upgrade everyone. There's currently an ecosystem split due to a new incompatible hyper, but it affects plenty of crates that just use basic reqwest::get or http::HeaderMap that haven't changed, and could be easily force-upgraded.

[kpreid]

I frequently wish for this ability, because of the following use case, which I have not seen mentioned already:

Your project depends on a package, and you want to test a change to that package which is only in its repository, not released. Depending on how the repository committers choose (or happen) to manage their Cargo.toml, this may be impossible without also forking, such as in these cases (I have encountered both):

• Minor/patch releases get version bumps in branches only, so the main branch has package.version = "1.2.0" even though version 1.2.3 is released on crates.io and required somewhere in your dependency graph. Here, there isn't a major version difference, but it is still incompatible.
• The major version is eagerly bumped after each release, i.e. you're using 0.24.0 and you want to test a patch, but the repository has package.version = "0.25.0" even though no breaking changes have yet been committed.

It would be particularly useful to reduce the effort required to patch in these cases, so that users of libraries can more quickly and easily test out changes and give feedback (e.g. to PRs actively in progress). Note that in this hypothetical, the repository is actively maintained — we're not patching stale/abandoned libraries — it's just that we don't want to give them additional tasks to do.

The way I would personally like to see this work is to just add a sledgehammer option to [patch] which replaces all versions of a package regardless of whether they are compatible. Possible syntax for the example in the original post:

[patch.crates-io]
openssl = { git = "https://github.com/sfackler/rust-openssl.git", patch-all-versions = true }

This would not work for all cases, but it would work for most of them (in my experience), and it would be faster to configure and have fewer edge-cases than a more precise "replace version 2.0.3 with version 3.0.0".

[kornelski]

I'm currently trying to upgrade plugins for Bevy 0.14, and this is causing so much pain.

Bevy has a huge graph of tightly coupled first-party and third-party crates depending on each other in specific major versions, and then depending on specific major versions of related crates like egui, wgpu, and winit.

Just updating one Bevy plugin requires replacing multiple crates and over 40 [patch.crates-io] entries! Even more annoyingly 0.14.0-dev in git is incompatible with 0.14.0-rc.2 on crates.io, so even crates already patched for 0.14.x need to be patched again to align their exact prerelease versions.

[John-Nagle]

I'm currently trying to upgrade plugins for Bevy 0.14, and this is causing so much pain.

Bevy has a huge graph of tightly coupled first-party and third-party crates depending on each other in specific major versions, and then depending on specific major versions of related crates like egui, wgpu, and winit.

That's where I am. 3D graphics involves many tightly coupled high performance moving parts. I'm trying to upgrade rend3, which also depends on specific versions of wgpu, egui, and winit. Plus minor stuff such as wgpu-profiling, naga, and glam. These are all maintained by different groups. Wgpu just released a major breaking change version, so there were weeks of issues and pull requests as everybody catches up. Then it turned out that the first breaking change caused a problem which required a second breaking change to wgpu, which restarted the whole churn cycle.

In this kind of situation, you need good version patching capability so you can test during the churn phase without too much extra work.

So that's the use case.

[Kixunil]

I have a case for this where there is a PR against the crate to update the major version (it's really only a change in Cargo.toml because the parts the library is using didn't break) but the maintainer is unavailable. And while an unavailable maintainer is a red flag the crate works fine and there isn't much that can go wrong anyway, so forking it just to change Cargo.toml seems excessive.

[kornelski]

There's a related problem of difficulty moving users of a library from 0.x to 1.0 when it goes "stable", without causing churn and fragmentation of an already de-facto stable final 0.x version.

https://github.com/rust-lang/cargo/issues/14460

I suspect both issues may have the same solution, which selectively allows unification of semver-major versions.