Wish: Support for cross-compilation

[vicuna]

Original bug ID: 4303 Reporter: n8gray Status: acknowledged (set by @xavierleroy on 2017-02-20T09:56:21Z) Resolution: open Priority: normal Severity: feature Version: 3.10.0 Category: platform support (windows, cross-compilation, etc) Related to: #6266 #6613 Monitored by: @ygrek @glondu Snark Camarade_Tux jm @hcarty "Richard Jones" @dra27 @avsm

Bug description

It would be nice if OCaml supported cross-compilation. There are a number of interesting platforms in existence today that are powerful enough to support an ocaml runtime environment but are not well suited for hosting the compiler itself, in particular mobile platforms (PocketPCs, Linux phones, Nintendo DS, PSP, etc). Cross-compilation is also attractive for Mac OS X users since it would allow us to build universal binaries of OCaml programs. There are various one-off patches floating around the net to get specific versions of OCaml to cross-compile to specific platforms, but it would be better to be able to build a single compiler supporting multiple architectures.

[vicuna]

Comment author: karl

Second this wish, I want ocaml for ARM platform.

[vicuna]

Comment author: @xavierleroy

On the CVS head version, the configure script was modified to allow specifying a cross C compiler and a cross assembler to be used. This removes one hurdle in building a Caml cross-compiler. I agree there should be a "HOWTO" explaining the steps to take. The major issue is the autoconfiguration stuff: either you manage to run it on the target device and copy the results to the build platform, or you have to write the configuration files by hand.

What you will never have, however, is a single OCaml compiler executable that can generate code for several target architectures.

[vicuna]

Comment author: Richard Jones

We are building a Windows cross-compiler environment as part of the Fedora MinGW project. You can find our test builds here:

http://hg.et.redhat.com/misc/fedora-mingw--devel/ (Click 'manifest' then one of the ocaml subdirectories)

[vicuna]

Comment author: @xavierleroy

Xavier Clerc is making progress on this issue, see:

http://brion.inria.fr/gallium/index.php/CrossCompiler

[vicuna]

Comment author: Camarade_Tux

I've noticed two iesues with this patch so far:

• doesn't really integrate with the build system
• seems unable to compile C (i.e. "ocamlc -c foo.c" will not work)

[vicuna]

Comment author: @xavierleroy

Un-assigning from xclerc

[vicuna]

Comment author: @damiendoligez

related to:

6266

940 https://github.com/ocaml/ocaml/pull/940

766 https://github.com/ocaml/ocaml/pull/766

620 https://github.com/ocaml/ocaml/pull/620

183 https://github.com/ocaml/ocaml/pull/183

[github-actions[bot]]

This issue has been open one year with no activity. Consequently, it is being marked with the "stale" label. What this means is that the issue will be automatically closed in 30 days unless more comments are added or the "stale" label is removed. Comments that provide new information on the issue are especially welcome: is it still reproducible? did it appear in other contexts? how critical is it? etc.

[nojb]

Still relevant.

[github-actions[bot]]

This issue has been open one year with no activity. Consequently, it is being marked with the "stale" label. What this means is that the issue will be automatically closed in 30 days unless more comments are added or the "stale" label is removed. Comments that provide new information on the issue are especially welcome: is it still reproducible? did it appear in other contexts? how critical is it? etc.

[shindere]

I didn't realise the issue was that old.

I am working towards this purpose for several years now.

Some people think I have made progresses in the right directions, some think differently.

Some people think I should stop working on this topic because, since I didn't manage to tackle it in so much time means I will never manage to, other people think differently and I hope the future will make them right.

The current plan on this to continue the work on the build system so that, ultimately, we have only one Makefile for the whole compiler. The expected outcome of that is twofold: easier maintainance of the build system and making that build system parallel by design and construction and not because the developers are clever enough to capture all the subtle dependencies among several makefiles (sometimes, even the most clever of them fail to do so and there is no reason not to provide all the dependency graph to the single instance of make).

Once this is done, hopefully in not too long, the next step will be to make the build system (and the compiler) less rigid about where the generated files go. this means, for instance, being able to compile the compiler out of its source tree. But it means also being able to have a clearer distinction between the artifacts produced by different compilers. At the moment, for example, files produced by boot/ocamlc and by ./ocamlc (rooot of source tree) go to the same place and are interleavedwith each other, which can create a lot of confusion. One visible symtom of this confusion in the build system is the partialclean target which, I think, would not be necessary if the artifacts of different compilers would be more clearly separated from each other.

Once we have all that in place, I think it will be come possible to start discussing cross-compilation on a sane and sound basis.

Assigning myself this issue and let's see how we can progress on that.

[nojb]

Sorry for the naïve question, but is there a roadmap that is being followed here? Right now, I get the feeling that these preliminary steps (single Makefile, out-of-tree compilation, etc) are nice-to-have but personally I don't have a clear picture of how everything is supposed to come together in the end.

In my opinion, having a clear HOWTO describing the roadmap would be most useful: are these intermediate steps (single Makefile, out-of-tree compilation) logically necessary for cross-compilation? What comes after? etc. Also, having such a roadmap would allow splitting the work into smaller pieces and allow other contributors to participate as well.

Judging by the fact that there are patches that achieve different forms of cross-compilation maintained by the community, it seems that some of these intermediate steps are not absolutely necessary. Can we take inspiration from these projects? If not, why not? What is missing from these projects? How do they work? etc. Having a document with all this information would already be a huge step forward.

Just my 2c.

[dra27]

It's a very reasonable 2¢, @nojb!

The main thing with a lot of the work towards this is that there are a lot of strands of work which pull in the same general direction. I often use the analogy of a ship-in-a-bottle, where at some point hopefully soon all the strings get gently pulled and we magically have a cross-compiler in the bottle - but at the moment it looks like a weird set of unrelated changes 🙂

There is a very limited roadmap from a few years ago in the developer-meetings repo which could be surfaced into a public RFC, yes. Based on my own experiments (February and June 2020), neither single Makefile nor out-of-tree builds are necessary for cross-compilation, but the result (certainly with in-tree builds) is considerably more complicated and involves a lot of filename mangling and dependency generation mangling.

For the existing solutions (I apologise in advance to their maintainers if my assessment is now out-of-date) - in particular, the opam-cross mechanisms build cross-compilers effectively by "tricking" our build system into using an existing host compiler (and opam-cross only has slower bytecode versions of the tools - i.e. there's no ocamlopt-compiled cross-compiler, unless that's changed). Both existing solutions (opam-cross and I think there's a version based on esy, but I'm not sure of the package names) rely on external systems to build the cross compilers (i.e. opam, esy, etc.). That said - and here's one of the strange unrelated strands - there's a possible benefit which @stedolan is investigating where being able to start-up the compiler's build using a pre-existing - and possibly older - compiler gives a boost to the build.

For the two things @shindere's actively working on now, I think it's worth differentiating "on the path to cross-compilation" from "worthwhile in their own right". For the unified Makefile work, again my own experiments trying to speed up the build of the compiler for opam users find multiple-minute savings just by tweaking the order in which we build things in the existing build infrastructure. However, those changes are hard to reason about, because the author is deciding that things can be safely built before other things, etc., rather than allowing the build system to know that via its dependency graph. The hunch is that with a unified Makefile, these performance boosts will be found and be easier to reason about. They certainly exist, because the Dune build of the compiler is much faster (not just the upstreamed one, but also from the flambda2 team's experience). Part of that work factors out build instructions., recipes, flags and so forth to one place - all of which need tweaking to be building cross-compilers.

For the out-of-tree builds, there are a few non-cross-compilation benefits. The bootstrap cycle becomes slightly more clear if it uses separate build trees for its phases (at present, it does a partialclean between each phase, saving the required artefacts from each stage in boot/ as it goes). We already have a partial out-of-tree build in that we maintain a partial compiler in boot/ and a full compiler in the root - both of which require myriad flags to execute and both of which could be made to execute "naturally" if combined with both my relocatable work and an out-of-tree ("installation") layout.

My own development preference for changes like this, as you might have noticed in the past, is huge numbers of commits, completely demonstrating the feature (vis-à-vis both "relocatable" and "no scripting" 🙂) but that comes with with two problems. Firstly, the work is terrifying to behold for reviewers (!!), and secondly, especially where the build system is concerned, it's exceptionally painful to rebase. This is why @shindere I think quite rightly prefers to be working on earlier stages as one project - it avoids the nasty rebases, but at the cost of not having a prototype in-tree cross-compiler to demonstrate the long-term benefit of the change.

[nojb]

Thanks for the explanations @dra27!

[jonahbeckford]

I think we can lift most of the material we'd need for a roadmap from what I wrote in https://diskuv.gitlab.io/diskuv-ocaml/doc/CompilingInDepth.html. That document was distilled into the dkml-base-compiler package in the opam repository, which supports cross-compiling macOS x86_64 <-> macOS arm64 and the various Android ABI cross-compiles (although I haven't done a public announcement).

Simplifying the cross-compiling procedure

The cross-compiling procedure is in https://diskuv.gitlab.io/diskuv-ocaml/doc/CompilingInDepth.html#changing-ocaml-to-do-cross-compilation

1. Just look at how often the procedure says "recompile" and "regenerate". This OCaml issue thread already mentioned the need to stop interleaving build artifacts so let's be explicit:
   • [ ] Roadmap: Stop interleaving build artifacts. A solution could be to provide "staged" build directories, where each build directory is focused on one phase (or stage) of the compilation.
2. This OCaml issue thread also mentioned simplifying down to a single Makefile. I didn't talk about that, but in dkml-base-compiler I had to write a bridge from CMake (the build system that Android uses) into ./configure. So for completeness I'll repeat it here:
   • [ ] Roadmap: Provide a single Makefile. The Makefile must be able to set the Target Machine configuration that is currently in utils/config.ml and runtime/sys.c

Removing Limitations

Each "limitation" can be a roadmap item. See the Limitations at https://diskuv.gitlab.io/diskuv-ocaml/doc/CompilingInDepth.html#limitations:

1. if your Target Machine is a 32-bit system, make sure you use a 32-bit Host Machine compiler. That equalizes [word_size]

   becomes

   • [ ] Roadmap: Solve the dual-purposes of word_size. A possible solution is to separate word_size into host_word_size and target_word_size

2. When ocamlopt is linking object files into an executable on Windows, it uses an executable called flexlink.exe that expects Windows .obj (COFF) object files for linking. However Linux uses ELF object files and macOS uses Mach-O object files, so a Windows Host Machine cannot support a non-Windows Target Machine. becomes

   • [ ] Roadmap: Solve the COFF dependency for flexlink.exe. A possible solution is to make flexlink.exe work with ELF and Mach-O object files

3. When ocamlopt is linking object files into an executable on Windows, the Host/Target Machine compiler must match (ie. MSVC or MinGW) and the Host/Target Machine word size (ie. 32 or 64) must match because flexlink.exe bundles a word size + compiler named object file flexdll_msvc.obj, flexdll_mingw64.obj, etc. into the final executable. becomes

   • [ ] Roadmap: Solve the flexdll Windows ABI dependency.

4. if your Target Machine is a Unix system, make sure you run the Host Machine cross-compiler on a Unix system. That equalizes ostype_unix

   and

   if your Target Machine is a Windows system, make sure you run the Host Machine cross-compiler on a Windows system. That equalizes ostype_windows

   becomes

   • [ ] Roadmap: Solve the ostype_unix and ostype_win32 and ostype_cygwin seperation.

[ChenQi1989]

@dra27 Sorry to bother. I've been looking at the configure and Makefile for some time. I found it's impossible to generate a cross-compiler (runs on linux x86-64 and generate codes for linux aarch64). The build process involves running 'ocamlrun', which is also expected to run on target. Am I missing something?

[shindere]

Chen Qi (2023/10/18 01:33 -0700):

@dra27 Sorry to bother. I've been looking at the configure and Makefile for some time. I found it's impossible to generate a cross-compiler (runs on linux x86-64 and generate codes for linux aarch64). The build process involves running 'ocamlrun', which is also expected to run on target. Am I missing something?

You are not missing anything. Quite the countrary: your observations are very accurate. One way or another, cross-comiling requires the ability to run two runtimes and, in theory, to build three (one for the build platform, one for the host platform where the compiler will be runningad one for the target platform where the generated programs will be running).

You are also right that such things are hard to impossible with the current build system. This is why (1) many projects to build cross-compilers are there but not of which has been upstremaed so far and (2) work is currently being done on the build system to make it less rigid, for instance in where the build artefacts are stored, so that it becomes possible to build several runtimes, among other things. AllthePRs that merge makefiles are there to make this kindof things easier, here again among other expected benefits.