Share some artifacts between `dune`'s contexts (cross-compilation)

I'm trying to have fun with cross-compilation, MirageOS and dune's contexts. And I just found a use that shouldn't be that specific when it comes to cross-compiling.

When building an artifact for another platform, we would like to include information that can only come from the host context (the default context) for a lot of reasons. A concrete example is the inclusion of a JavaScript file which should be obtained using js_of_ocaml and which could only be built (for obvious reasons[^1]) in the host context.

I think it should be possible to use this artifact to obtain another one that would only be available in a specific and different context. In another example, it may be of interest to include information about the host context in which the binary was cross-compiled. This would involve generating OCaml code in the default context and being able to integrate it into the cross-compiled binary (i.e. in another context).

That's why I talk about sharing artefacts between contexts. It can be interesting to get for a binary from context A, some (OCaml) files from context B. There is already a solution with the variable {exe which always refers to an output from the default context. However, as far as MirageOS is concerned, we don't have the latitude to really introspect the dependencies necessary to obtain a cross-compiled binary and thus explicitly say that our binary depends on a source from another context.

Let's take a little example:

we have our default context and another toolchain (like Solo5, you can install it via opam install ocaml-solo5)
we want to generate a public.ml from the default context
however, we want to generate main.exe from our cross-compiler context (our Solo5 toolchain)
and our main.exe depends on our public.ml

Such as:

$ cat >dune <<EOF
(executable
 (name main)
 (modules :standard)
 (enabled_if (= %{context_name} "solo5")))

(rule
 (target public.ml)
 (action (with-stdout-to %{target} (run ./generate/generate.exe)))
 (enabled_if (= %{context_name} "default")))
EOF
$ mkdir generate
$ cat >generate/dune <<EOF
(executable
 (name generate)
 (modules generate)
 (enabled_if (= %{context_name} "default")))
EOF
$ cat >generate/generate.ml <<EOF
let () = Format.printf {ocaml|let string = "Hello World!"|ocaml}
EOF
$ cat >dune-project <<EOF
(lang dune 3.0)
EOF
$ cat >dune-workspace <<EOF
(lang dune 3.0)

(context (default))

(context
 (default
  (name solo5)
  (host default)
  (toolchain solo5)
  (disable_dynamically_linked_foreign_archives true)))
EOF
$ cat >main.ml <<EOF
let () = print_endline Public.string
EOF
$ dune build
File "main.ml", line 1, characters 23-36:
1 | let () = print_endline Public.string
                           ^^^^^^^^^^^^^
Error: Unbound module Public

The real error here is about (modules :standard) where dune should be able to infer that a public.ml is needed to compile the main.exe binary. A rule to produce it exists only on the %{context_name} = "default" but it seems that dune is not able to do that. It will be really nice to add the ability to dune to search some required artifacts from some other contexts or more generally, be able to cross-use some artifacts over contexts.

[^1]: When I say obvious reason, it's mostly because producing a JS file with js_of_ocaml would come from a js_of_ocaml available from the default toolchain. We would not want to depend on a js_of_ocaml available in our cross-compilation context for example which is more limited or only seeks to produce another assembler.

ocaml / dune

Share some artifacts between `dune`'s contexts (cross-compilation) #5592