Problem compiling with threads

[dbuenzli]

The problem is that:

> ocamlfind query -predicates native -format "%d/%a" threads

> ocamlfind query -predicates byte -format "%d/%a" threads

Reading through .opam/$SWITCH/lib/threads/META it seems we have to specifiy either threads.posix or threads.vm I infer that ocamlfind must do some magic when it linkpkgs threads.

[dbuenzli]

Ok so the problem is that we need to add mt,mt_posix or mt,mt_vm predicates. Ideally I guess this should flow down from having a Flags.thread in the context, but the current resolver doesn't have that kind of context.

I will just hack something that adds these predicates whenever it sees the threads package. I think the package resolution strategy needs refactoring anyways, see comment in #89.

[dbuenzli]

Mmmh the hack doesn't work if threads is a recursive dep.

[dbuenzli]

Argh. I'm not getting anywhere with that one. Basically after having considered puking-prone solutions inside assemblage itself. I thought I would in fact be sufficient to make it clear in the docs that binaries using the -vmthread flag must have a dependency on threads.vm, and those with -thread flag must have a dependency on threads.posix. The flags and the dep would then naturally trickle down to units, compiling them with the environment :

open Assemblage

let units = `Units [ unit "main" (`Path ["."]) ]
let posix =
  let deps = [ pkg "threads.posix" ] in
  bin "hello-pthread" ~deps ~flags:Flags.thread units

let vm =
  let deps = [ pkg "threads.vm" ] in
  bin "hello-vmthread" ~deps ~flags:Flags.vmthread ~native:false units

let () = assemble (project "hello-thread" [posix; vm])

and problem solved. But no, for the following reason:

>  ocamlfind query -r -predicates byte -format "-I %d" -r threads.posix
-I /usr/local/lib/ocaml
-I /usr/local/lib/ocaml/threads

Now when ocamldep looks for dependencies it sees the thread.mli in the first include and generates:

_build/bin-hello-pthread/main.cmo : /usr/local/lib/ocaml/thread.cmi
_build/bin-hello-pthread/main.cmx : /usr/local/lib/ocaml/thread.cmi

but there is no such cmi, as it is in the directory of the second include.

I don't know which tool is more broken but basically it seems to be impossible to replicate the magic that ocamlfind does when you call ocamlfind tool by simply using -predicate.

Why is all this so broken ?

[dbuenzli]

Ok so two things had to be done. I describe them here for documentation purposes:

First in units we no longer track external (package) dependencies (f93386f0df02037bf). This solves the highlighted in my preceeding comment with ocamldep.

But then there was some more brokenness at link time, which can be seen by the following invocations, the first two calls is what we'd get without changing our system, the answers don't make sense, the two subsequent calls is what is needed. So there's now a hack in the As_ocamlfind resolver that adds the mt, mt_vm or mt_posix predicates to the request whenever the request for packages includes the threads.vm or threads.posix packages (ab5c6f4c1668055922bc5b65).

> ocamlfind query -r -predicates byte -format "%d/%a" -r threads.vm
/usr/local/lib/ocaml/unix.cma
> ocamlfind query -r -predicates byte -format "%d/%a" -r threads.posix
/usr/local/lib/ocaml/unix.cma
> ocamlfind query -r -predicates byte,mt,mt_vm -format "%d/%a" -r threads.vm
/usr/local/lib/ocaml/vmthreads/unix.cma
/usr/local/lib/ocaml/vmthreads/threads.cma
> ocamlfind query -r -predicates byte,mt,mt_posix -format "%d/%a" -r threads.posix
/usr/local/lib/ocaml/unix.cma
/usr/local/lib/ocaml/threads/threads.cma

Note that in general it doesn't seem to be possible to code against a threads backend in an agnostic manner since all sources need to be compiled with -thread or -vmthread and the cmis for each backend are different (see manual).

Also note that there is some magic in the compiler itself here that will change the includes accordingly if -thread or -vmthread is on the command line. This means that for compiling itself specifying these flags may be enough (no package dependencies needed) but it's better if we depend on the package. I also have the impression that these flags do only that (change the includes) in that case we should get rid of them I think and simply use correct ocamlfind answers derived by querying the packages threads.posix or threads.vm. I asked the list here.

[samoht]

I guess this is now fixed by your patches. Thanks! (see #99)