Add --toolchain option, for ocamlbuild -toolchain

[whitequark]

Depends on https://github.com/ocaml/ocamlbuild/pull/125, fixes https://github.com/dbuenzli/topkg/issues/86.

[dbuenzli]

I don't have this in my head but why can't this be simply defined by a configuration key, e.g. like in 2bf815d3784c726529964 and appropriate lookup in Pkg.build_cmd ? Especially one baked by an environment variable which would allow to reuse topkg OPAM files without any change in cross-compilation repos.

[whitequark]

I don't have this in my head but why can't this be simply defined by a configuration key, e.g. like in 2bf815d and appropriate lookup in Pkg.build_cmd ?

No reason other than that I find the code of the new topkg very hard to follow. TOPKG_TOOLCHAIN sounds fine?

Especially one baked by an environment variable which would allow to reuse topkg OPAM files without any change in cross-compilation repos.

This doesn't work on opam 1.x and I think it didn't properly work in opam 2.x either last time I checked that thread.

[dbuenzli]

TOPKG_TOOLCHAIN sounds fine?

Rather TOPKG_CONF_TOOLCHAIN

[whitequark]

Done, and works in opam-cross-*

[whitequark]

Ping?

Also you'll have to update whatever process by which the natdynlink capability gets discovered to work with -toolchain, as this is another instance in which I found topkg too opaque to patch properly.

[dbuenzli]

Also you'll have to update whatever process by which the natdynlink capability gets discovered to work with -toolchain, as this is another instance in which I found topkg too opaque to patch properly.

I don't exactly what you see as obscure here. But in any case topkg precisely allows to make a difference between build and host tools. This is described here with the lookup procedure in which you can hook so that that your binaries point to the right one. I'm actually surprised that you didn't try to use this rather than enshrine ocamlfind -toolchain approach which now bleeds into ocamlbuild aswell. You may want to review this ocamlbuild churn in light of the above capabilities offered by topkg (which I'll gadly adjust if they are found not to be suitable enough).

In any case, regarding natdynlink. The OCaml configuration is looked up using ocamlc -config using this mecanism, this is described here. Since the natdynlink capability is not exposed yet in the configuration (see https://github.com/ocaml/ocaml/pull/970), it is discovered as described here, namely we look for the existence of the dynlink.cma file in the standard library returned by ocamlc -config), the corresponding code is here. If you correctly allow the lookup procedure described above to point to the right ocaml host and build os binaries, everything will work.

[whitequark]

This is described here with the lookup procedure in which you can hook so that that your binaries point to the right one.

Thanks, the item #4 shows exactly the change that I need to perform:

Cmd.(v "ocamlfind" % "mytool") if "mytool" is part of the OCaml tools that can be invoked through ocamlfind.

This should be trivial to implement now.

[whitequark]

I'm actually surprised that you didn't try to use this rather than enshrine ocamlfind -toolchain approach which now bleeds into ocamlbuild aswell.

BTW I am very happy about having ocamlfind -toolchain as well as (now) ocamlbuild -toolchain. This makes it far easier to work with the majority of packages, which do not use topkg; for example I think (but haven't checked yet) that with the ocamlbuild patch, cross-compilation could be achieved for oasis packages with a single configure.ml option.

Eventually I would like to generate cross-packages based on host-packages using a simple script instead of the current manual approach. Of course, first-class opam support would be better but I do not spend enough time changing them manually to justify implementing that and the opam team does not appear to have a strong interest in cross-compilation, seeing as to how little discussion https://github.com/ocaml/opam/issues/2476 has generated.

[whitequark]

@dbuenzli I have updated the PR to address your concern about overriding build_cmd, and verified that it works with my opam-cross-* repositories.

One unresolved question is: what do we do with the ocamlbuild dependency specification? Currently it's a {build} one. In my opinion it is incorrect because topkg invokes ocamlbuild.

• One option is to just remove {build}. Then, no upgrades will be necessary, and the users of the --toolchain option (which after all is the domain of the repository and not package itself) could add the stricter dependency themselves.
• Another option is to replace it with {>= 0.11.0} (once 0.11.0 gets released). This is more proper in my view, but would mandate some upgrades.

[dbuenzli]

Looks right, but could you please document and expose the configuration key to the end users API around here. Also the documentation of the default build command here should be updated.

You are right about the fact that build is inappropriate. I think we should simply drop it and add a {>= 0.11.0}. However which upgrades do you see happening ? Normally topkg packages only build depend on ocamlfind, ocamlbuild and topkg so I don't see the upgrades you see happening, topkg will simply impose a constraint on the version of ocamlbuild which I don't think should be a problem for the packages out there.

[whitequark]

However which upgrades do you see happening ? Normally topkg packages only build depend on ocamlfind, ocamlbuild and topkg so I don't see the upgrades you see happening, topkg will simply impose a constraint on the version of ocamlbuild which I don't think should be a problem for the packages out there.

I mean existing installs with pre-0.11.0 ocambuild will have to be upgraded, potentially triggering a lot of rebuilds.

[whitequark]

Looks right, but could you please document and expose the configuration key to the end users API around here. Also the documentation of the default build command here should be updated.

Done.

[dbuenzli]

Thanks ! Your patch is in as 05685ce095c88721e731c. You may want to keep your fork around until a release is done because the current master depends on an unreleased version of cmdliner.

[whitequark]

@dbuenzli ack. There is also no ocamlbuild release to require as an opam dependency, yet.