[native-image] Cross compilation support?

[sureshg]

Any plans to add cross-compilation feature to native-image? I know it's too early to ask for these kind of requirements since we don't even have a working Windows build. IMHO, this would be a nice feature (golang has nice support for it) to have especially if we want to build binaries for all platforms from a CI (Jenkins) machine.

https://medium.com/@chrisgseaton/i-dont-think-we-have-any-immediate-plans-for-cross-compilation-but-it-s-not-a-major-problem-to-bf789384beaa

[plombardi89]

Definitely would love to see this feature! This would make it so much nicer to use Java for CLI apps and servers that need to be distributed to users on Linux, Mac and Windows.

[alza-bitz]

It would be amazing to have cross-compilation support. Is this still not planned for yet, or is it dependent on the Windows build? It would still be useful to have cross-compilation available to create Mac binaries on a (Linux) CI server, and then add support for Windows cross-compilation later?

[alza-bitz]

@thomaswue @chrisseaton I was just wondering if this is on the roadmap for 2019, or has any ETA etc? I think it would be a game-changer!

[alza-bitz]

@thomaswue @chrisseaton I was just wondering if this is on the roadmap for 2019, or has any ETA etc? I think it would be a game-changer!

Hi @thomaswue @chrisseaton since we're halfway through the year, I just thought I'd check in on this one..

Has there been any development on this issue, or any news/progress to report?

Thanks in advance! 🙂

[sureshg]

@thomaswue Now we have java 11 and windows support ready, are there any plans to consider cross-compilation support in the near future?

[thomaswue]

Adding the support is quite tricky. The issue is that there are parts of the JDK initialized during the native image generation process that could be platform dependent and then end up in the image. One of the alternatives could be to offer a service for creating native images. Also, maybe one could use maybe a system like GitHub actions to build images for different platforms without the need to maintain/setup those machines. Would this help for your use cases or is true cross compilation a requirement?

[xiaodong-xie]

I do think people should have mentioned this workaround elsewhere.

I am using this https://hub.docker.com/r/oracle/graalvm-ce/ docker image, to build the native-image running on Linux, on my MacBook laptop.

[sureshg]

Would this help for your use cases

@thomaswue Thanks. Yeah, GitHub action would work for OSS projects. The issue is, our build system is mostly on Linux and wants to build native-images for windows and mac (mostly for CLI apps). So the cross-compilation would have definitely helped in this scenario.

[nhoughto]

Yeah it’s not really cross compilation if you need the target platform to build it, that’s just regular old compilation. If Cross compilation is a feature being targeted all the jvm weirdness will need to be solved, I guess the question is, is/when is it a priority?

[sureshg]

Here is some discussion on the current limitations - https://graalvm.slack.com/archives/CN9KSFB40/p1582755160011700

[truh]

Here is some discussion on the current limitations - https://graalvm.slack.com/archives/CN9KSFB40/p1582755160011700

Would it be possible to share a summary of the discussion? The slack archive can only be accessed by people that are member to the workspace.

[sureshg]

@truh

Christian Wimmer : 
No, cross compilation between different OS is more or less impossible in
 the current approach because OS specific JDK classes need to be loaded 
in the image generator

The biggest problem is, e.g., the file system and network stack. The JDK code
 for that is very different for Linux, MacOS, and Windows. You cannot load the 
code from two platforms at the same time, but you need to load the code from 
the platform that the image generator runs on.

[CircuitRCAY]

Anything on cross compiling between architectures? (x86_64 -> arm64, for example)

[8Keep]

This is pretty huge for me. Is there any future plan at all to find a way to do this?

[SaraDark]

+1000 to Cross compilation support

[tuhuynh27]

+10000 to Cross compilation support

[rmannibucau]

+1, it is particularly interesting for arm64 (pi) support where relying on (general) amd64 power to build would be a great enhancement.

[bademux]

@truh

Christian Wimmer : 
No, cross compilation between different OS is more or less impossible in
 the current approach because OS specific JDK classes need to be loaded 
in the image generator
....

WINE is our friend, cross compiling for windows is a big deal. Unfortunately native-image.exe and gu.exe are failing on WINE 6\ WINE7 with 0100:fixme:virtual:NtAllocateVirtualMemoryEx Ignoring 1 extended parameters 0x11fae0

upd. Steps To reproduce:

1. Build dockerfile below `docker build -t graalvm-win .
2. Run it docker run -it graalvm-win bash -c "wine graalvm-ce-java17-22.0.0.2/lib/installer/bin/gu.exe -h"

   
   FROM ubuntu:21.10

RUN apt-get update \ && apt-get install -y ca-certificates software-properties-common winbind curl unzip \ && apt-get clean -y && rm -rf /var/lib/apt/lists/*

RUN curl --fail --silent --location --retry 3 https://dl.winehq.org/wine-builds/winehq.key | apt-key add - \ && add-apt-repository 'deb https://dl.winehq.org/wine-builds/ubuntu/ impish main' \ && dpkg --add-architecture i386 && apt-get update \ && apt install -y --install-recommends xvfb winehq-devel \ && apt-get clean -y && rm -rf /var/lib/apt/lists/*

WORKDIR /opt

ARG GRAALVM_VERSION=22.0.0.2 ARG JAVA_VERSION=java17 ARG GRAALVM_ARCH=windows-amd64 ARG GRAALVM_PKG=https://github.com/graalvm/graalvm-ce-builds/releases/download/vm-$GRAALVM_VERSION/graalvm-ce-$JAVA_VERSION-$GRAALVM_ARCH-$GRAALVM_VERSION.zip ARG TARGETPLATFORM=linux/amd64

ENV LANG=en_US.UTF-8 \ JAVA_HOME=/opt/graalvm-ce-$JAVA_VERSION-$GRAALVM_VERSION

RUN curl --fail --silent --location --retry 3 ${GRAALVM_PKG} -o /opt/graalvm.zip && unzip -q /opt/graalvm.zip -d /opt && rm /opt/graalvm.zip

RUN wine64 wineboot --init && while pgrep wineserver > /dev/null; do sleep 1; done

CMD java -version

[rmannibucau]

I was expecting the cross-compilation to be possible (when user code is 100% java) by precomputing most of the code-graph in the distro and just importing the result (so, dll, etc included) in native-image main. Can be a saner path than wine which is proven to be quite dependent of the env from my experience.

[bademux]

@rmannibucau it is impossible. You have to have Win build toolchain and Windows build toolchain is not crossplatform, but it works on wine (VS2019 tested). It means we can build container with all tools needed for windows build.

The only blocker is that GraalVM tools ( native-image.exe and gu.exe) don't work with WINE. I will be grateful for any help.

[rmannibucau]

@bademux I understand but it means you generate sources and not .so/.dll directly whereas it is possible to generate .so/.dll directly. That said I'm fine with a cross platform toolchain while it does not rely on wine (once again I know it works on your computer, will likely work on a few others but can trivially break with any upgrade of distro so it is way too fragile for a build chain). My preference would be to rely on light windows docker image finely tagged (versions) so this is reproducible for most people and can even use native windows tools. Maybe @microsoft could help with that?

[bademux]

@rmannibucau it is not entirely true, you pure java code still use platform specific jvm binaries underneath. 2nd - docker is answer for what you call fragility - just freeze you dependencies in container. As well you can use easy-peasy multiplatform docker build with docker buildx, so multiple hardware platforms are covered. 3rd - ms would help here if they allow to redistribute BuildTools by changing license, so you can actually share docker image not a recipe (Dockerfile).

And last but not least and being realistic - it will be easy to solve just one blocker towards build on Windows, then just waiting for "ideal" solution.

[rmannibucau]

@bademux

it is not entirely true, you pure java code still use platform specific jvm binaries underneath.

yes but while the codebase on top of the jvm (understand jre excluded) does not it is fine

Agree on the other point but it should be made easier thanks to graalvm docker images IMHO, currently it is a lot of custom steps and builds and this is what we can simplify a lot IMHO.

[ppalaga]

So cross-platform seems to be rather impossible. Is there at least any hope for cross-arch compilation, like osx@amd64 -> osx@aarch64?

[koutheir]

So cross-platform seems to be rather impossible.

Cross-compilation using native-image is not impossible. We already do it using our own fork of Graal, which allows us to cross architectures on the same OS.

Is there at least any hope for cross-arch compilation, like osx@amd64 -> osx@aarch64?

We already build native images for linux/aarch64 and linux/armv7a on a machine running linux/amd64.

[mixaal]

@koutheir , could you please share more info on the cross-compilation? It'd help me a lot. At least in the same OS, different architecture case. Thanks a lot !

[bademux]

We already build native images for linux/aarch64 and linux/armv7a on a machine running linux/amd64.

It is an easy task 😜 to compile multiarch on Linux. It is doable right now without any forks, just use docker multiarch build infrastructure. Give me true cross platform, like compiling static win binary on Linux/osX.

[koutheir]

@mixaal, I can't really share how we do it, because it is part of a proprietary product. I simply wanted to inform you that it is possible. I'm sorry.

@bademux, we do real cross-compilation, without any containers or VMs. Doing cross-OS native-image generation is simply the next step on our way if a customer needs that.

[bademux]

@koutheir sounds like big work here. I'm not sure how static binary for win supposed to be cooked on Linux\osX, but finger crossed. Please share!

There is a difference between "it is possible to jump 15m height" and "it is possible to jump to the moon". For now both options is equally impossible until someone invest $.

I can't see the difference if real and imaginary (?) crosscompilation produce the same result.

[rmannibucau]

Did you investigate mingw? It is often used for C/C++ projects so even if it can require to not go directly to .so it can be an option maybe using C/C++ as intermediary language?

[chrisseaton]

I'm not sure how static binary for win supposed to be cooked on Linux\osX

Binaries are just files with numbers in them. You can produce those numbers on any platform. Cross-compilation shouldn't be a weird thing.

[bademux]

@chrisseaton I'm not sure what to do with linking for Win static binaries on osX\linux, but anyway someone should invest time in creating tooling and support it. That is why I see more perspective in WINE + docker, then dreaming about cross platform (cross os) compilation.

[koutheir]

@koutheir sounds like big work here.

I never said it was little work. I only said it is possible, and done.

I'm not sure how static binary for win supposed to be cooked on Linux\osX, but finger crossed.

It is possible if you follow the GNU ABI and use a mingw64 linker.

Please share!

I can't. I'm sorry.

[t-oster]

@koutheir you source code is proprietary, but wouldn't it be possible to just share your steps for building the executable? I think a Docker-Container which just compiles a simple Hello-World app to a native Windows-Exe would be awesome and could be used as a base-image for many people, trying the same. I understand you have a closed-source fork of GraalVM for it, but are your modifications all necessary? Maybe you could convince your employer to just contribute the parts necessary for building cross-platform images.

[koutheir]

but wouldn't it be possible to just share your steps for building the executable?

From the point of view of a Java developer, we provide a custom native-image that produces native images for linux/armv7a on a linux/amd64. It's not a multi-step process for the Java developer.

I think a Docker-Container which just compiles a simple Hello-World app to a native Windows-Exe would be awesome and could be used as a base-image for many people, trying the same.

Again, we don't use containers or virtual machines. We modify Graal far enough to support building native images for linux/armv7a on a linux/amd64.

I understand you have a closed-source fork of GraalVM for it, but are your modifications all necessary?

Yes, they are, and they are substantial.

Maybe you could convince your employer to just contribute the parts necessary for building cross-platform images.

My employer keeps that as a commercial secret.

[t-oster]

sorry to hear that. I know you're not using containers, but in order to share a setup with all dependencies installed, a Docker image would be a good option IMHO. However if you're unable to share your code, our only hope is other developers who come up with a solution. Thanks for your quick response.

[d9j]

it's 2022 still not cross compile support. really guys? with such steps hope in 2025 it will finally arrive

[koutheir]

If Graal remains focused on servers, cross-compilation might never be implemented, because the assumption will be that the target computer will always be good enough to compile Graal itself and generate native images that it will run. If Graal starts targeting embedded systems, then cross-compilation will be one of the first things to implement.

[rmannibucau]

@koutheir not sure this reasoning works because it assumes you deliver a native image of an application you own - path to do it for a vendor is way too hard as of today, in particular to maintain and validate. So if not possible it means CLI/embedded will get a harder time than needed but also that native will also be harder for products (external apps, including oss). So hope it gets enabled for both cases.

[thomaswue]

Our GitHub Action for GraalVM supports creating native images for different platforms simultaneously. Here is an example configuration to create an image each for MacOS, Windows, and Linux: https://github.com/marketplace/actions/github-action-for-graalvm#building-a-helloworld-with-graalvm-native-image-on-different-platforms

There can be differences between operating systems how the JDK or third-party library dependencies are initialized and those end up snapshotted into the native image. Therefore, the problem of cross compilation for native image is more difficult to solve than cross compilation for e.g. C programs.

We plan to provide cross-compilation support for different architectures - i.e., amd64/aarch64. There are currently no plans however to provide such cross-compilation support for different operating systems - i.e., creating a Windows binary from Linux or vice versa. In general our strategy to simplify creation of native images goes more into the direction of providing native image generation services where one can specify the desired architecture.

[xlight05]

We plan to provide cross-compilation support for different architectures - i.e., amd64/aarch64. There are currently no plans however to provide such cross-compilation support for different operating systems - i.e., creating a Windows binary from Linux or vice versa. In general our strategy to simplify creation of native images goes more into the direction of providing native image generation services where one can specify the desired architecture.

@thomaswue Do we have any timelines or ETA for providing cross compilation for different architectures?

[thomaswue]

What is the primary source/destination architecture you would be interested in to cross-compile, is it Linux/amd64 to Linux/aarch64? Is it possible to solve your use case with the GitHub Action workaround?

[nyetwurk]

I would like to build installers (or, worst case, single binaries) for windows/amd64, linux/amd64, darwin/amd64, darwin/arm64 all from a single platform, preferably linux/amd64

For windows, historically i've used launch4j + nsis For linux, a tar file is more than sufficient (since you can trivally start a jar from a shell script) For darwin/MacOS jpackage sort of works but not really

[bademux]

@thomaswue I'm interested in Win64, linux cross arch compilation is easy achievable with docker cross build (buildx)

[carlosedp]

I've been involved in multiple architecture porting projects and the biggest use-case is cross-building Linux amd64 -> arm64 and the other way around (arm64 -> amd64) where the first one is most used since there are biggest/fastest amd64 boxes. If possible, cross-OS building would be desireable like linux/amd64 -> darwin/amd64 as Golang currently does (it actually does any cross-building like linux/amd64 -> darwin/arm64 and so on).

[chabala]

Still hoping for sparc.

[rmannibucau]

Hi,

from Linux/amd64 to windows|arm64 for me.

Any CI is not an option on some env so would be great to have a solution close to c++ solutions.

[xlight05]

What is the primary source/destination architecture you would be interested in to cross-compile, is it Linux/amd64 to Linux/aarch64? Is it possible to solve your use case with the GitHub Action workaround?

@thomaswue

I think the real advantage of cross compilation comes when JVM languages/frameworks adopt GraalVM Ex - Quarkus. Lets assume I wanna deploy an azure function using my framework(which is a perfect usecase where AOT shines). As far as I know, Azure functions only supports executable with linux/amd64. The language/framework can wrap graalvm and generate that executable for the user by hiding that complexity from user.

So the github actions workaround would not work in this case as it needs to happen from the framework's side, not the users side. The closest workaround i have is building it using docker using buildx. The issue with this is the docker dependency. Still, even with docker, its hard to do the cross compilation for different operating systems.

[brainchild0]

For enterprise deployments, it may suffice in many contexts for a vendor to utilize a server for building each new release of a project. However, cross building is extremely standard for deployments of open-source platforms and packages. It is hard to see how a tool would gain much traction for open source projects these days if it constrains the target and build environment to be the same platform or machine family.

[thomaswue]

@brainchild0 Agreed. We believe however that many open source (and even commercial) projects can use our GitHub Action for GraalVM to build their releases. Is this not a reasonable workaround? https://github.com/marketplace/actions/github-action-for-graalvm

We could also maybe build an Oracle cloud service for building.