Closed edwintorok closed 4 years ago
That would be useful indeed; I'll try to do something for that. Note that we can't get this out of the solver, and that the reasons printed are already computed a posteriori by a heuristic
I'm not sure if this is related or a separate bug, but if I have depext 0.3 installed and pin to a new 0.4 dev version, the downgrade reason is bogus:
$ opam pin add depext git://github.com/avsm/opam-depext#fix-detection
depext is now git-pinned to git://github.com/avsm/opam-depext#fix-detection
[depext] git://github.com/avsm/opam-depext#fix-detection updated
[depext] Installing new package description from git://github.com/avsm/opam-depext#fix-detection
depext needs to be reinstalled.
The following actions will be performed:
- install ocamlfind 1.5.5 [required by cmdliner]
- downgrade cmdliner 0.9.7 to 0.9.2 [required by depext]
- upgrade depext 0.3 to 0.4*
===== 1 to install | 1 to upgrade | 1 to downgrade =====
=-=- Gathering sources =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
[cmdliner.0.9.2] http://erratique.ch/software/cmdliner/releases/cmdliner-0.9.2.tbz downloaded
[depext] git://github.com/avsm/opam-depext#fix-detection already up-to-date
[ocamlfind.1.5.5] http://download.camlcity.org/download/findlib-1.5.5.tar.gz downloaded
=-=- Processing actions -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
-> removed depext.0.3
-> removed cmdliner.0.9.7
-> installed ocamlfind.1.5.5
-> installed cmdliner.0.9.2
-> installed depext.0.4
Done.
opam@b5caedeaff59:~/opam-repository$ opam --version
1.2.1
opam@b5caedeaff59:~/opam-repository$ opam upgrade
The following actions will be performed:
- upgrade cmdliner 0.9.2 to 0.9.7
- recompile depext 0.4* [uses cmdliner]
===== 1 to recompile | 1 to upgrade =====
=-=- Gathering sources =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
[cmdliner.0.9.7] http://erratique.ch/software/cmdliner/releases/cmdliner-0.9.7.tbz downloaded
[depext] git://github.com/avsm/opam-depext#fix-detection already up-to-date
=-=- Processing actions -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
-> removed depext.0.4
-> removed cmdliner.0.9.2
-> installed cmdliner.0.9.7
-> installed depext.0.4
Done.
An opam pin -n
followed by opam install
does not do this downgrade.
@avsm hm, weird indeed... would be interesting to see the cudf. This was with aspcud 1.9 ?
Yes this was on MacOS X. I dont have the CUDF available, but it was from a clean switch and so should be reproducible by trying a depext pin + version bump I hope.
(It showed up in the Docker builds)
Here is another situation where downgrade reasons would be helpful: https://gist.githubusercontent.com/edwintorok/e49f47f308d3eb842af8/raw/00e24a4c8b02f376d1c4399bf75e7fdd0b9f21c6/fixup-1.cudf
[WARNING] A conflict was detected in your installation. This can be caused by updated constraints or conflicts in your installed packages:
- Conflicting version constraints for bin_prot
- Conflicting version constraints for textutils
- No package matches core.<unavailable version>.
- No package matches core_kernel.<unavailable version>.
- No package matches custom_printf.<unavailable version>.
- core_kernel<111.25.00 is not available because it requires OCaml >= 4.01.0 & < 4.02.1.
- custom_printf<111.21.00 is not available because it requires OCaml >= 4.00.0 & < 4.02.0.
The following dependencies are in cause:
- core_bench -> bin_prot <= 112.17.00
- core_bench -> core = 109.35.00 | core = 109.40.00 | core = 109.47.00 | core = 109.53.01 | core = 109.55.00 | core = <unavailable version> | core <= 109.31.00 | core <= 109.38.00 | core <= 109.45.00 | core <= 109.55.02 | core <= 110.01.00 -> pa_test = 109.53.00 -> core_kernel < 111.25.00
- core_bench -> core = 109.35.00 | core = 109.40.00 | core = 109.47.00 | core = 109.53.01 | core = 109.55.00 | core = <unavailable version> | core <= 109.31.00 | core <= 109.38.00 | core <= 109.45.00 | core <= 109.55.02 | core <= 110.01.00 -> custom_printf = <unavailable version> | custom_printf < 111.21.00
- core_bench -> textutils = 109.24.00 | textutils <= 111.28.00 | textutils <= 112.01.00 | textutils <= 112.06.00 -> core = 109.35.00 | core = 111.03.00 | core <= 109.31.00 | core <= 109.47.00 | core <= 109.55.00 | core <= 110.01.00 | core <= 111.21.00 -> core_kernel = <unavailable version>
- core_bench -> textutils = 109.24.00 | textutils <= 111.28.00 | textutils <= 112.01.00 | textutils <= 112.06.00 -> core = 109.35.00 | core = 111.03.00 | core <= 109.31.00 | core <= 109.47.00 | core <= 109.55.00 | core <= 110.01.00 | core <= 111.21.00 -> custom_printf = <unavailable version> | custom_printf < 111.21.00
- core_bench -> textutils = 109.24.00 | textutils <= 111.28.00 | textutils <= 112.01.00 | textutils <= 112.06.00 -> core = 109.35.00 | core = 111.03.00 | core <= 109.31.00 | core <= 109.47.00 | core <= 109.55.00 | core <= 110.01.00 | core <= 111.21.00 -> pa_test = 109.53.00 | pa_test = 110.01.00 -> core_kernel < 111.25.00
You should run "opam upgrade --fixup" to resolve the situation.
$ opam upgrade --cudf=fixup --fixup
The following actions will be performed:
↘ downgrade bin_prot 112.24.00 to 112.17.00 [required by core_bench, async_kernel, core, core_kernel]
↘ downgrade sexplib 112.24.00 to 112.17.00 [required by uri, textutils, mirage-types-lwt, etc.]
↘ downgrade pa_ounit 112.24.00 to 112.17.00 [required by core_kernel, core, async_kernel, etc.]
↘ downgrade typerep 112.24.00 to 112.17.00 [required by core_kernel]
↘ downgrade pa_test 112.24.00 to 111.08.01 [required by core, async_kernel, core_kernel]
↻ recompile uri 1.8.0
↻ recompile ipaddr 2.6.1 [uses sexplib]
↻ recompile ezjsonm 0.4.1
↻ recompile cstruct 1.5.0 [uses sexplib]
↻ recompile cconv 0.3.1
↻ recompile camlhighlight 3.0 [uses sexplib]
↘ downgrade custom_printf 112.24.00 to 112.17.00 [required by core]
↻ recompile pa_bench 112.06.00 [uses pa_ounit]
↻ recompile tuntap 1.0.0 [uses ipaddr]
↻ recompile ocsigenserver 2.5 [uses ipaddr]
↻ recompile cow 1.2.1
↻ recompile xenstore 1.2.5
↻ recompile tar-format 0.2.1 [uses cstruct]
↻ recompile nbd 1.0.2 [uses cstruct]
↻ recompile mstruct 1.3.2 [uses cstruct]
↻ recompile io-page 1.2.0 [uses cstruct]
↻ recompile asn1-combinators 0.1.1 [uses cstruct]
↘ downgrade core_kernel 112.24.00 to 112.17.00
↻ recompile aws 0.0.3
↻ recompile xenstore_transport 0.9.4 [uses xenstore]
↻ recompile mirage-types 2.2.0 [uses sexplib]
↻ recompile mirage-profile 0.4 [uses cstruct]
↘ downgrade core 112.24.00 to 112.17.00
↻ recompile vhd-format 0.7.1
↻ recompile mirage-types-lwt 2.2.0
↻ recompile mirage-net-unix 2.2.0
↻ recompile mirage-clock-xen 1.0.0 [uses mirage-types]
↻ recompile mirage-clock-unix 1.0.0 [uses mirage-types]
↻ recompile mirage-block-unix 1.2.2 [uses cstruct]
↻ recompile mirage 2.2.1
↻ recompile mbr-format 0.2 [uses cstruct, ipaddr]
↻ recompile xen-gnt 2.2.0
↻ recompile shared-memory-ring 1.1.1 [uses cstruct]
↘ downgrade async_kernel 112.24.00 to 112.17.00
↻ recompile textutils 112.17.00
↻ recompile opass 1.0.2
↻ recompile mirage-fs-unix 1.1.4
↻ recompile fat-filesystem 0.10.2 [uses cstruct]
↻ recompile mirage-xen 2.2.2
↻ recompile mirage-unix 2.2.2
↘ downgrade core_bench 112.24.00 to 112.17.00
↻ recompile vchan 2.0.2
↻ recompile mirage-net-xen 1.4.1
↻ recompile ctypes 0.4.0 [uses mirage-xen]
↻ recompile mirage-console 2.1.3
↻ recompile tsdl 0.8.1
↻ recompile sodium 0.4.0*
↻ recompile nocrypto 0.3.1 [uses mirage-xen]
↻ recompile iocaml-kernel 0.4.6
↻ recompile tcpip 2.3.0 [uses mirage-unix, mirage-net-unix, mirage-console, etc.]
↻ recompile x509 0.2.1 [uses sexplib]
↻ recompile otr 0.1.0
↻ recompile erm_xmpp 0.3 [uses nocrypto]
↻ recompile dns 0.14.0
↻ recompile tls 0.3.0
↻ recompile mirage-dns 2.0.0 [uses dns, mirage-types-lwt]
↻ recompile libres3 1.0
↻ recompile conduit 0.7.2
↻ recompile mirage-conduit 2.0.0
↻ recompile cohttp 0.15.0 [uses conduit, uri]
↻ recompile xen-api-client 0.9.8 [uses uri]
↻ recompile mirage-http 2.1.0
↻ recompile cowabloga 0.0.9
↻ recompile xe-unikernel-upload 0.4
===== ↻ 59 ↘ 10 =====
Do you want to continue ? [Y/n]
The following dependencies are in cause:
These long disjunctions can be shortened a lot, so that it gets a bit more readable. There is code for that in OWS, I believe, I need to backport it: should yield something like
The following dependencies are in cause:
- core_bench -> bin_prot <= 112.17.00
- core_bench -> core <= 110.01.00 -> pa_test = 109.53.00 -> core_kernel < 111.25.00
- core_bench -> core <= 110.01.00 -> custom_printf = <unavailable version> | custom_printf < 111.21.00
- core_bench -> textutils <= 112.06.00 -> core <= 111.21.00 -> core_kernel = <unavailable version>
- core_bench -> textutils <= 112.06.00 -> core <= 111.21.00 -> custom_printf = <unavailable version> | custom_printf < 111.21.00
- core_bench -> textutils <= 112.06.00 -> core <= 111.21.00 -> pa_test = 110.01.00 -> core_kernel < 111.25.00
It seems indeed some reasons are missing in the list of actions, too, besides printing additional version constraints. It would be helpful if you still had the fixup-actions.dot
file (generated along fixup-1.cudf
.
Thanks!
Here are all the fixup*
files:
git clone https://gist.github.com/c1254104fda5d235d2ad.git
Awesome, thanks!
For the last example, it's very difficult, because in case of a fixup
, we have no way to know the roots of the action (when you do an opam install
, causality is computed from the packages you specified). The root cause may actually not appear at all in the graph (e.g. some packages are removed because they conflict with foo
, but foo
itself is unchanged).
What is done currently is that we take the actions of input degree 0 and compute from there. I have already much enhanced the display (uncommited yet) so that it doesn't miss relationships (for some reasons, it computed from upgrades but not downgrades). But for now you'll still not get the reason for e.g. downgrade core_bench
-- we don't know!
That could be added by an additional mechanism (the current one only relies on the action graph) that checks remaining unexplained nodes for compatibility with the current state and prints the offending constraints. Since that would only be useful for fixup
, it may actually be simpler to just display the conflicts before the solution.
To get back to the original issue, it may be interesting, in the case of a command with specified packages, to filter the reasons for a version change to the ones that are really related to the version change: at present the "causes" only refer to package relationships, not actual actions relationships:
downgrade ctypes 0.4.0 to 0.3.4 [required by tgls]
means that ctypes is required by tgls, not that tgls requires the downgrade
@avsm I could reproduce a similar scenario using the internal solver. I guess finding meaningful explanations to the bogus actions of the internal solver is beyond the scope of my little explanation heuristics. (I may want to start writing a Lawyer
module for that)
Also, see the end of the long post above: when it says
- install ocamlfind 1.5.5 [required by cmdliner]
- downgrade cmdliner 0.9.7 to 0.9.2 [required by depext]
- upgrade depext 0.3 to 0.4*
it means that cmdliner
is required by depext
, and that ocamlfind
is required by cmdliner
(0.9.2). Which is right. But doesn't explain a lot :)
On 03/27/2015 10:55 AM, Louis Gesbert wrote:
For the last example, it's very difficult, because in case of a |fixup|, we have no way to know the roots of the action (when you do an |opam install|, causality is computed from the packages you specified). The root cause may actually not appear at all in the graph (e.g. some packages are removed because they conflict with |foo|, but |foo| itself is unchanged).
What is done currently is that we take the actions of input degree 0 and compute from there. I have already much enhanced the display (uncommited yet) so that it doesn't miss relationships (for some reasons, it computed from upgrades but not downgrades). But for now you'll still not get the reason for e.g. |downgrade core_bench| -- we don't know!
That could be added by an additional mechanism (the current one only relies on the action graph) that checks remaining unexplained nodes for compatibility with the current state and prints the offending constraints. Since that would only be useful for |fixup|, it may actually be simpler to just display the conflicts before the solution.
To get back to the original issue, it may be interesting, in the case of a command with specified packages, to filter the reasons for a version change to the ones that are really related to the version change: at present the "causes" only refer to /package/ relationships, not actual /actions/ relationships:
downgrade ctypes 0.4.0 to 0.3.4 [required by tgls] means that ctypes is required by tgls, not that tgls requires the downgrade
Yep, the 'required by' is great when the action is install, although it'd probably be confusing to change its semantics now.
I thought that you could say 'conflicting with ..., keeping due to ...', or 'due to constraints in ..., needed by ...' but it might get too verbose.
Perhaps just using the unicode pictograms would work, and using 'requested by' for the action, and 'required by' for listing reverse dependencies:
Here the two needs to be separated as the 'required by' are dependencies, and the keep is reverse dependencies
A package is listed in the 'required by' only if that change actually caused the install/downgrade/removal action like you said, i.e. just because a reverse-dep gets upgraded/downgraded it doesn't mean the arrow should be displayed in required by, it should be displayed only if the previously installed version didn't require this action to be taken, and the newly installed one does. The '*' should be easier as it should always be traceable back to an install root requested on the cmdline. The '?' would be displayed if the reverse-dep is just a depopt and wouldn't cause the install on its own.
Not sure whether packages should be listed twice (both in requested by and required by), at least for the install action that'd look odd, so perhaps install should be kept closer to current situation
(unicode pictogram only shown if that action triggered the install):
install: [required by
Not sure how to represent that the action is (or was) a direct cmdline action (opam install/opam remove), I marked this with
On 03/27/2015 10:55 AM, Louis Gesbert wrote:
What is done currently is that we take the actions of input degree 0 and compute from there. I have already much enhanced the display (uncommited yet) so that it doesn't miss relationships (for some reasons, it computed from upgrades but not downgrades). But for now you'll still not get the reason for e.g. |downgrade core_bench| -- we don't know!
I think the reason is "upstream changes": core_bench.112.24.00 doesn't exist (it was removed from the opam repository), which brings along all the core downgrades and recompilations.
I think that
I think that
in opam's output is a hint to that (or to opam's confusion), except its core_bench itself that is unavailable not the other ones.
This one should no longer appear since my latest patches, I now pass along the version maps, allowing the original referred to version to be printed.
I am beginning to see an algorithm that could improve, but that could get quite complicated -- actually, it already is, trying to cover a graph in a custom order, extracting likely reasons depending on node relationships -- so it should probably be done as a separate pass, either before or after (can't decide yet). For upgrade, downgrade or removal actions, we would take the version that used to be installed and check:
foo
that aren't satisfied by the old version: ">= 1.0 Required by bar.2"The current algorithm processes only on the action graph, so doesn't have (direct) access to the package universes, availability information, dependencies or conflicts.
Also note that we may still be interested in relations that are not of real causality, in the case of an opam upgrade
without arguments for example: foo
may be upgraded just because of the optimisation criteria, but is it still worthwhile to display that it depends on bar
even if the upgrade of bar
didn't really force the upgrade of foo
?
opam (both 1.2.0 and 1.2.1-rc2) gave me the solution below for a package install. Initially I thought there is something wrong with the solver, but turns out it has to choose between somewhat equivalent situations:
Could opam print some hints on relevant package constraints for the downgrade?
After it has the solution it could check where the downgrade constraints come from, and print it for example:
cudf files available here just in case: cudf downgrade log downgrade cudf recompile log recompile