samoht
commented
4 years ago Looks great. I guess multiple opam calls is a bit slow, but it's better to be precise.
Closed hannesm closed 4 years ago
samoht
commented
4 years ago Looks great. I guess multiple opam calls is a bit slow, but it's better to be precise.
hannesm
commented
4 years ago it actually turns out that multiple calls without --rec are "just file reads", while a call with --rec involves the CUDF solver (which takes quite some time). (all AFAIU, I don't know the internals too well)
rjbou
commented
4 years ago @hannesm
As you noted, packages are not computed the same way: --required-by is map/list manipulation while with --rec it is using the cudf graph (not the solver). These two commands should return a common subset, unless there is choices (|) in dependencies.
On the loaded repositories, if --no-switch is not given, switch state is loaded with all its recorded repositories. And the packages selection is done on repositories packages and installed ones (with or without depopts).
Feel free to open an issue if there is incoherent/incorrect behaviors :) It can also help other that have the same questions
samoht
commented
4 years ago Unfortunately this still doesn't seem to work properly.
$ opam list --installed -s --required-by fmt.0.8.8
ocaml
ocamlbuild
ocamlfind
seq
stdlib-shims
topkgbut
$ opam install fmt.0.8.8
$ opam list --installed -s --required-by fmt
ocaml
ocamlbuild
ocamlfind
result
seq
stdlib-shims
topkg
uchar@samoht maybe dune-build-info will solve the job for us soon? :)
hannesm
commented
4 years ago since we're querying only the package name (without version) and have it installed, it may be an overapproximation of the included packages -- which is fine with me since build dependencies are in there as well.
samoht
commented
4 years ago It's just not very reproducible, the result is different in the CI and on my machine (which is annoying). I'll check dune-build-info but in the meantime I've pushed https://github.com/mirage/mirage/pull/1042/commits/c41e02fdfa7c1cbf6ff40bb8cf9ffe098a5945f2
rjbou
commented
4 years ago @samoht --installed is a filter on the resulting package list, not a selector on the given package. Then the first command gives the list of all dependencies of all versions of fmt that are installed in the switch, while the second one restrain it to that 0.8.8 version.
It is more visible on the not shortened output
$ opam list --installed --required fmt
# Packages matching: installed & required-by(fmt)
# Name # Installed # Synopsis
ocaml 4.07.1 The OCaml compiler (virtual package)
ocamlbuild 0.14.0 OCamlbuild is a build system with builtin rules to easily build most OCaml projects.
ocamlfind 1.8.1 A library manager for OCaml
result 1.4 Compatibility Result module
seq base Compatibility package for OCaml's standard iterator type starting from 4.07.
stdlib-shims 0.1.0 Backport some of the new stdlib features to older compiler
topkg 1.0.1 The transitory OCaml software packager
uchar 0.0.2 Compatibility library for OCaml's Uchar module
$ opam list --installed --required fmt.0.8.8
# Packages matching: installed & required-by(fmt.0.8.8)
# Name # Installed # Synopsis
ocaml 4.07.1 The OCaml compiler (virtual package)
ocamlbuild 0.14.0 OCamlbuild is a build system with builtin rules to easily build most OCaml projects.
ocamlfind 1.8.1 A library manager for OCaml
seq base Compatibility package for OCaml's standard iterator type starting from 4.07.
stdlib-shims 0.1.0 Backport some of the new stdlib features to older compiler
topkg 1.0.1 The transitory OCaml software packager
While testing reproducible builds of my MirageOS unikernels with
orbin a modified version, which does the following steps:yyyto build a unikernel (== opam package)yyyknows about a list of opam repositories provided on command-lineopam exportNow, the goal is to reproduce the very same (bitwise equal) binary. This is achieved by
orb rebuild:yyy(using the same BUILD_PATH as before)opam importthe exported switch (this installs the unikernel as well)That works reliable, the issue (fixed in this PR) is that
opam list --installed --rec <pkgs>returns different results in the first and second build (I suspect since the--recleads to a code path in opam that uses thecudfsolver (which seems to not know about packages from other repositories than the default repository in the second build). This PR avoids using--rec, but does the recursion/fixpoint in functoria. When--recis not used, only the opam file is consulted, no cudf involved --> same result and reproducibility.As further information (//cc @AltGr @rjbou to take a brief look whether this is an opam issue or not -- my intuition is that
list --required-by yyshould always be a subset oflist --rec --required-by yy) -- this is the imported switch above, i.e. it has all the packages installed, but only knows about the "default" repository:With
--rec, there are more packages, but some are missing (conduit-lwt/awa-mirage):