Pass custom dependency to `opam2nix.build`?

[kamilchm]

Hi,

I'm trying to use opam2nix.build with specs of libraries depends on cmdliner version from outside of opam like:

let cmdliner = (opam2nix.buildOpamPackage {
    name = "cmdliner-8500634a";
    src = fetchFromGitHub {
      owner = "esy-ocaml";
      repo = "cmdliner";
      rev = "8500634a96019c4d29b1751628025b693f2b97d6";
      sha256 = "0dgc6dhwfvghism99v0bbbdmzs6kmsa6kgb25zjm740n3k87wlb7";
    };
  });
in
opam2nix.build {
  specs = [ { name = "logs"; } { name = "fmt"; } ];
}

to enable compilation of logs.cli and fmt.cli. But I don't know how to pass my cmdliner to opam2nix.build. Is there a way to do it?

[timbertson]

Yep! I do exactly this myself in passe: https://github.com/timbertson/passe/blob/7c369277a55149987c7bc26efc9cfd923620c323/nix/opam-deps.nix#L15-L81

build and friends take an override attribute, which is a fix-point function (much like nixpkgs overlays).

The main dfference is:

• the arguments are {self, super}: instead of self: super: (so you can't get the order wrong)
• they override what's called the "opam2nix world", which is not well documented. But some relevant attributes are:

opamPackages: attrset of all packages and versions, e.g. opamPackages.cmdliner."1.2.3" opamSelections: attrset of selected package versions, e.g. opamSelections.cmdliner (which is normally a reference to a specific version in opamPackages.

So either you can override specific versions if you have a relevant patch for some versions, or you can just override "the selected version" if you just want to make sure your version is used.

[kamilchm]

I can't make it work :( I try like that https://github.com/kamilchm/nix-esy/blob/master/default.nix#L40-L42 Do you have an example simpler than the passe one?

[dysinger]

@timbertson the order is opposite {self, super}: (in the above comment) vs {super, self}: (in the linked source code in passe)

[timbertson]

@dysinger That's the point - I never remember which order they're supposed to go so I pass them as named attributes (where the order doesn't matter) instead of ordered arguments ;)

As for the code, this line looks wrong:

opamPackages = self.opamPackages // { cmdliner = { "1.0.2-8500634a" = cmdliner; }; };

You can't use self.opamPackages // ... since that's infinitely recursive (you're defining self right now). Using super.opamPackages // ... should work.

What that should result in is an infinite recursion error. If not, that implies it's not actually evaluating your override's opamPackages attribute. What error do you actually see, @kamilchm ?

I cloned it locally, and I get:

building '/nix/store/n4y8aswsscz0g4lpc6g3fvabfag5p38i-opam-selection.nix.drv'...
+ env OCAMLRUNPARAM=b /nix/store/sxlzd7vz3rfcwa845a8mkhkz25y2szhy-opam2nix-0.5.0/bin/opam2nix select --repo /nix/store/dqgp08fiqxhx1a7rk1nby0kbip0wr5rf-opam2nix-generated-packages --dest /nix/store/1274xiklhpdlnhkhk6mb7dw9lix8i949-opam-selection.nix --ocaml-version 4.06.1 --base-packages base-unix,base-bigarray,base-threads --ocaml-attr ocaml astring=0.8.3 base-bigarray=base base-bytes=base base-threads=base base-unix=base base=v0.11.1 bigstringaf=0.4.0 biniou=1.2.0 bos=0.2.0 camomile=1.0.1 conf-m4=1 conf-perl=1 conf-which=1 cppo=1.6.5 cppo_ocamlbuild=1.6.0 cudf=0.9 dose3=5.0.1 dune=1.6.3 easy-format=1.3.1 extlib=1.7.5 fieldslib=v0.11.0 fmt=0.8.5 fpath=0.7.2 jbuilder=transition lambda-term=1.13 logs=0.6.2 lwt=4.1.0 lwt_log=1.1.0 lwt_ppx=1.2.1 lwt_react=1.1.1 menhir=20181113 merlin-extend=0.3 merlin=3.2.2 ocaml-compiler-libs=v0.11.0 ocaml-migrate-parsetree=1.2.0 ocamlbuild=0.12.0 ocamlfind=1.8.0 ocamlgraph=1.8.8 opam-core=2.0.2 opam-file-format=2.0.0 opam-format=2.0.2 opam-repository=2.0.2 opam-state=2.0.2 ppx_assert=v0.11.0 ppx_compare=v0.11.1 ppx_custom_printf=v0.11.0 ppx_derivers=1.0 ppx_deriving=4.2.1 ppx_deriving_yojson=3.3 ppx_expect=v0.11.0 ppx_fields_conv=v0.11.0 ppx_here=v0.11.0 ppx_inline_test=v0.11.0 ppx_let=v0.11.0 ppx_sexp_conv=v0.11.2 ppx_tools=5.1+4.06.0 ppx_tools_versioned=5.2.1 ppx_variants_conv=v0.11.1 ppxfind=1.2 ppxlib=0.5.0 re=1.8.0 react=1.2.1 result=1.3 rresult=0.6.0 seq=0.1 sexplib0=v0.11.0 stdio=v0.11.0 topkg=1.0.0 uchar=0.0.2 utop=2.2.0 variantslib=v0.11.0 yojson=1.5.0 zed=1.6 reason angstrom cmdliner=1.0.2-8500634a
+ /nix/store/sxlzd7vz3rfcwa845a8mkhkz25y2szhy-opam2nix-0.5.0/bin/opam2nix select --repo /nix/store/dqgp08fiqxhx1a7rk1nby0kbip0wr5rf-opam2nix-generated-packages --dest /nix/store/1274xiklhpdlnhkhk6mb7dw9lix8i949-opam-selection.nix --ocaml-version 4.06.1 --base-packages base-unix,base-bigarray,base-threads --ocaml-attr ocaml astring=0.8.3 base-bigarray=base base-bytes=base base-threads=base base-unix=base base=v0.11.1 bigstringaf=0.4.0 biniou=1.2.0 bos=0.2.0 camomile=1.0.1 conf-m4=1 conf-perl=1 conf-which=1 cppo=1.6.5 cppo_ocamlbuild=1.6.0 cudf=0.9 dose3=5.0.1 dune=1.6.3 easy-format=1.3.1 extlib=1.7.5 fieldslib=v0.11.0 fmt=0.8.5 fpath=0.7.2 jbuilder=transition lambda-term=1.13 logs=0.6.2 lwt=4.1.0 lwt_log=1.1.0 lwt_ppx=1.2.1 lwt_react=1.1.1 menhir=20181113 merlin-extend=0.3 merlin=3.2.2 ocaml-compiler-libs=v0.11.0 ocaml-migrate-parsetree=1.2.0 ocamlbuild=0.12.0 ocamlfind=1.8.0 ocamlgraph=1.8.8 opam-core=2.0.2 opam-file-format=2.0.0 opam-format=2.0.2 opam-repository=2.0.2 opam-state=2.0.2 ppx_assert=v0.11.0 ppx_compare=v0.11.1 ppx_custom_printf=v0.11.0 ppx_derivers=1.0 ppx_deriving=4.2.1 ppx_deriving_yojson=3.3 ppx_expect=v0.11.0 ppx_fields_conv=v0.11.0 ppx_here=v0.11.0 ppx_inline_test=v0.11.0 ppx_let=v0.11.0 ppx_sexp_conv=v0.11.2 ppx_tools=5.1+4.06.0 ppx_tools_versioned=5.2.1 ppx_variants_conv=v0.11.1 ppxfind=1.2 ppxlib=0.5.0 re=1.8.0 react=1.2.1 result=1.3 rresult=0.6.0 seq=0.1 sexplib0=v0.11.0 stdio=v0.11.0 topkg=1.0.0 uchar=0.0.2 utop=2.2.0 variantslib=v0.11.0 yojson=1.5.0 zed=1.6 reason angstrom cmdliner=1.0.2-8500634a
WARN: opam var enable-ocaml-beta-repository not found...
Assuming package ocaml-beta is unavailable due to error: Failure("Undefined boolean filter value: enable-ocaml-beta-repository")
Your request can't be satisfied:
  - package cmdliner version >1.2 unavailable

It looks like you're combining esyDeps with cmdliner=1.0.2-8500634a. From the error, it sounds like something in esyDeps has a transitive dep on cmdliner >1.2, so your added constraint on a specific version makes it impossible to solve.

The reason your overrides hasn't (yet) been evaluated is because there's actually two phases going on - the selections phase, and the build phase. Selections only uses a small amount of information (what's documented for opam2nix.selectionsFile in the README), since all it really needs is all the opam files in the repo, plus the list of packages + constraints (specs) you want to solve for.

A caveat of this is that if you need to select a specific version of cmdliner, that version must be in the repo. It looks like your custom version has a hash in it, so I assume that version is not present in the opam-repository. So even if this version was acceptable given your constraints, the solver wouldn't know about it. There's two solutions there:

• if the dependencies for your cmdliner don't differ from 1.0.2 on which it's based, you can just pin your constraint to cmdliner=1.0.2 (so it can be selected), and then override that version (in opamPackages) with your custom implementation.
• if your fork of cmdliner has different dependencies from the real 1.0.2, the solver will need to actually use that version of the opam file instead of just pretending it's equivalent to upstream's 1.0.2. You can pass extraRepos to make additional package definitions available to the solver, which you can make with buildOpamRepo. Hopefully you don't need this though, it's a bit more complex.

(sorry this is so complex, if there's a simpler way to arrange all of this I haven't stumbled on it yet)

[kamilchm]

Thanks a lot! Changing the constraint to 1.0.2 helped.