astral-sh / uv

An extremely fast Python package and project manager, written in Rust.
https://docs.astral.sh/uv
Apache License 2.0
27.77k stars 797 forks source link

Allow dependency name overriding or elimination #4422

Open leorochael opened 5 months ago

leorochael commented 5 months ago

While uv now has overrides, and issue #2686 is suggesting being able to declare certain overrides as pertaining to a specific dependency, I have an additional suggestion/request.

I would like to be able to override a package dependency with a package that has a different name.

As a concrete case, the FuelSDK package depends on suds-jurko, but has been tested to work with it's fork/successor project suds-community, a.k.a. just suds these days.

(Yes, the project has come full circle, with the fork of the fork officially taking over the original name).

A possibility that comes to mind would be a special syntax for the -o overrides.txt file like:

suds-jurko=>suds>=1.1,<2.0

Or perhaps:

suds-jurko:suds>=1.1,<2.0

Considering the long history of python packages that fork and continue the work of orphaned packages (e.g. PIL/Pillow), this could be a nice complement to the overrides functionality of uv.

Perhaps it could even be used to remove a dependency completely, e.g.:

leorochael commented 5 months ago

Thinking about this some more, just the functionality of blocking a dependency from being installed already offers a way to do this kind of override.

That is, I could add to my requirements.in:

Fuel-SDK
suds>=1.1,<2.0

Then to my overrides one of the options for deleting a removing a dependency I mentioned previously, or perhaps just a special version. e.g one of:

zanieb commented 5 months ago

Interesting idea! Makes some sense, though I'm also not quite sure how it should be expressed and if it's feasible.

leorochael commented 5 months ago

Another option to express dependency elision is to use the URI version specifier to mean: install this package from nowhere.

The last one was suggested to me by github copilot:

To explicitly reference nothing or create a URI that signifies an absence of a resource, you can use the about:blank URI. This URI leads to an empty document in web browsers and is often used to represent a null or non-existent resource in contexts where a URI is required but there is no actual resource to point to.

leorochael commented 5 months ago

And since URIs effectively allow namespacing on schemes, and the definition of the interpretation of schemes is up to whoever introduces them, we could perhaps use schemes to implement the package replacement, e.g:

leorochael commented 5 months ago

Or, if we want to follow examples of other tools which use prefixed schemes, like git+, we could namespace the scheme with uv+, like:

leorochael commented 5 months ago

Note that --no-emit-package/--unsafe-package suds-jurko doesn't help here, because suds-jurko is broken at package building, so resolution still fails.

leorochael commented 5 months ago

An alternative that would help me would be to just ignore the sub-dependencies of a package on a per-package basis.

That is, instead of having an elision or renaming on my overrides file, I would like to add something like this in my requirements.in:

# FuelSDK has broken sub-dependencies:
--no-deps-for FuelSDK
FuelSDK
# Corrected FuelSDK dependencies manually added here:
pyjwt >= 1.5.3
requests >= 2.18.4
suds >= 1.1.2

For reference, such an option has been under discussion for a while in pypa/pip#9948

charliermarsh commented 5 months ago

This seems like a reasonable use-case to me.

hauntsaninja commented 4 months ago

I think this would be very useful! At work, we have systems like this for first party packages, but not third party packages. It would make some things less painful if this were the case and unblock some other future use cases.

A related lower priority feature request would be the ability to override edges, not just nodes. E.g., if I know one package is overpinning a dependency in a way that I know is safe to override, I might still want to resolve older or error if something else pins it.

I don't have strong opinions on syntax. => is a good suggestion, but could be typo-ed. Maybe using @@ as a separator? lhs is the thing to be overriden, rhs is empty or PEP 508

node>=2
node @@ node>=2  # same as above
node @@ node_nextgen>=2  # replace node with node_nextgen
node @@  # eliminate node (and don't resolve its deps)
node -> dep @@ dep>=2  # override node's dependency on dep

If syntax is controversial, another option is to switch to TOML or something, especially if uv has plans to use such a format somewhere else.

charliermarsh commented 4 months ago

I realized that this actually is possible today... You can use a never-truthy marker. For example, to remove typing-extensions, use an overrides.txt like:

typing-extensions ; sys_platform == 'never'
charliermarsh commented 4 months ago

(This works because with overrides, we just replace all requirements of typing-extensions with whatever is in the overrides file.)

leorochael commented 4 months ago

(This works because with overrides, we just replace all requirements of typing-extensions with whatever is in the overrides file.)

I'm surprised this works, because I'd expect it would result in the resolution being unsolvable...

And I'm slightly suspicious of relying on it and it being just accidental behaviour that might be "fixed" in the future...

charliermarsh commented 4 months ago

@leorochael -- Can you say more about why you would expect that to be unsolvable? I would actually consider it a bug if the resolver failed there, rather than the other way around.

leorochael commented 4 months ago

I might be reading it wrong, but for me an overrides file containing:

Mean:

And if there are packages depending on typing-extensions, in my mind that would mean their dependencies would be unsolvable because of that.

To me, declaring a constraints override on an uninstallable package is different than declaring a constraints override on a package we're pretending is already installed.

leorochael commented 4 months ago

Ah, but I see where my logic is failing.

The constraints override doesn't say: "depend on this uninstallable package".

It's saying: "when depending on typing-extensions, only do that if the platform is never", which never happens.

So, yeah, as long as we add a test for that and document it, then yes, that would already be supported.

hauntsaninja commented 4 months ago

Oh, neat trick!