search

JuliaLang / Pkg.jl

Pkg - Package manager for the Julia programming language
https://pkgdocs.julialang.org
Other
620 stars 260 forks source link

Old Transducers package installed for unknown reasons #3198

Open BioTurboNick opened 2 years ago

BioTurboNick commented 2 years ago

I have an issue where Pkg is preferring to install an old package version, but there does not seem to be any reason why.

After running up and running st --outdated -m, I have Transducers.jl at v0.1.0, when the current version is v0.4.73. It indicates that the package is upgradable (green caret), but the yellow text indicates it is constrained to <v0.4.73, but does not indicate any package causing this to be the case.

image

This is in the global environment, on Julia 1.8.1. The Project.toml file has no compat entries.

Is it possible to find out more about where this constraint came from? And/or why it went all the way back it v0.1.0?

BioTurboNick commented 2 years ago

Possibly related to #3119

KristofferC commented 2 years ago

Is it possible to find out more about where this constraint came from?

Try pkg> add Transducers@0.4.73.

BioTurboNick commented 2 years ago

Pkg installs it without complaint. Except Compat.jl downgrades from v4.2.0 to v3.46.0 And now all the outdated entries show a yellow bounded caret.

And now it shows me that Compat is constrained to <v4.2.0 due to ArrayInterface v3.2.2 ("3.37") and BangBang ("2.0, 3.0"). (Shouldn't it say it is constrained to <v4.0.0?)

ArrayInterface (current version is 6.x) is in turn also constrained by TiledIteration ("3"), although Pkg doesn't show that as the constraint? It shows <v6.0.23, but it should probably show <v4.0.0?

A tangled web of possibly outdated compat entries to figure out, but those aside, shouldn't Pkg be attempting to eliminate green carets when update is run?

KristofferC commented 2 years ago

Except Compat.jl downgrades from v4.2.0 to v3.46.0

So you have two valid resolved states and Pkg will pick one of them. If you need to disambiguate you can specify [compat].

shouldn't Pkg be attempting to eliminate green carets when update is run?

Well, not if it downgrades another package :P. Green carets doesn't mean that the packages are guaranteed to be upgraded on pkg> upgrade, it means there is nothing explicitly holding them back in the current environment.

This is basically dup of https://github.com/JuliaLang/Pkg.jl/issues/3150.

BioTurboNick commented 2 years ago

Is it really a dup? The carets seem to be accurate. It is possible to update those packages as there are no constraints among the direct dependencies of the environment. It's just that the indirect dependency Compat happens to be updated first, so it is given precedence over later indirect dependencies. Right?

I would rather have dependencies higher up in the chain be prioritized. Compat is a tertiary dependency, so it seems wrong for it to be controlling here.

But absent that, I would prefer to not have Pkg flip flop between states. E.g. update produces one valid configuration, update SpecificPackage produces another valid configuration, and then update flips it back. If the order of update results in a constrained later package, it would be better IMO to have update SpecificPackage error and provide instruction for how to resolve the situation.

KristofferC commented 2 years ago

Pkg should give you a pareto optimal point. If you need further disambiguation you can use [compat].

BioTurboNick commented 2 years ago

Okay. The flipping back and forth without an indicator of why is still a problem IMO.

Something like @info "Packages <list1> were not brought current because they would cause Packages [list2] to downgrade, due to update execution order. If you would like to ensure [list1] packages are prioritized, you may need to add a [compat] entry to the appropriate Package.toml. would be useful.

And then something like @warn Package <name1> was previously constrained due to update execution order. The update was successful, but a general update may downgrade this package again. You may need to add a [compat] entry to the appropriate Package.toml. could be useful.

BioTurboNick commented 1 year ago

Just saw a report of a similar issue in Slack, with RelocatableFolders being downgraded from 1.0.0 to 0.1.3, breaking PackageCompiler with WGLMakie.

Part of the issue here is that package versions without any [compat] exist in the registry, so there's always something to fall back to.

Is there a way to prevent this kind of breaking fallback?

KristofferC commented 1 year ago

breaking PackageCompiler with WGLMakie.

Why did it break?

Part of the issue here is that package versions without any [compat] exist in the registry,

Most of that should have been fixed with https://github.com/JuliaRegistries/General/pull/11114.

Is there a way to prevent this kind of breaking fallback?

This isn't really a fallback, it is just normal package resolution. You can add [compat] entries in your Project or ensure that package versions with bad compat info are yanked or possibly "retro capped" similar to the PR I linked above.

BioTurboNick commented 1 year ago

I'm helping get more information from them.

Some interaction between Pluto, RelocatableFolders, PackageCompiler. I presume others in the environment may also be involved.

The break here is just unexpected downgrades without a clear reason.

KristofferC commented 1 year ago

The reason is probably that some other package would have to downgrade otherwise.