IT test fails when building native image using -Dquarkus.native.container-build=true in MacOS

[faskan]

Describe the bug

IT test fails with below error when building native image using "quarkus build --native -Dquarkus.native.container-build=true". The project is a hello world project generated using quarkus-amazon-lambda-rest-archetype with no additional changes.

[ERROR] Tests run: 4, Failures: 0, Errors: 1, Skipped: 3, Time elapsed: 2.084 s <<< FAILURE! - in com.techroots.GreetingIT [ERROR] com.techroots.GreetingIT.testFunqy Time elapsed: 0.01 s <<< ERROR! java.lang.RuntimeException: Unable to successfully launch process '32277'. Exit code is: '126'.

[INFO] [INFO] Results: [INFO] [ERROR] Errors: [ERROR] GreetingIT.testFunqy » Runtime Unable to successfully launch process '32277'. ... [INFO] [ERROR] Tests run: 4, Failures: 0, Errors: 1, Skipped: 3 [INFO] [INFO] [INFO] --- maven-failsafe-plugin:3.0.0-M5:verify (default) @ pdfgenerator --- [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 03:04 min [INFO] Finished at: 2022-04-10T23:26:23+02:00

Expected behavior

IT test should not fail

Actual behavior

GreetingIT fails

How to Reproduce?

1. Generate a new maven project using mvn archetype:generate \ -DarchetypeGroupId=io.quarkus \ -DarchetypeArtifactId=quarkus-amazon-lambda-rest-archetype \ -DarchetypeVersion=2.7.5.Final
2. quarkus build --native -Dquarkus.native.container-build=true

Output of uname -a or ver

Darwin Faisals-MacBook-Pro.local 21.2.0 Darwin Kernel Version 21.2.0: Sun Nov 28 20:28:54 PST 2021; root:xnu-8019.61.5~1/RELEASE_X86_64 x86_64

Output of java -version

No response

GraalVM version (if different from Java)

No response

Quarkus version or git rev

2.7.5.Final

Build tool (ie. output of mvnw --version or gradlew --version)

No response

Additional information

No response

[quarkus-bot[bot]]

/cc @matejvasek, @patriot1burke

[matejvasek]

@faskan just guessing here: -Dquarkus.native.container-build=true will run build in container, docker on macOS usually limits memory of each container to something around 2-3GiB AFAIK and native build consumes a lot of memory. Although OOM usually returns different error code (137).

[matejvasek]

Does it only happen with -Dquarkus.native.container-build=true flag?

[matejvasek]

@faskan another speculation: since you are doing build in container the resulting binary might be of different os/architecture. The container probably produces Linux's ELF, but your mac expects Mach-O.

[matejvasek]

@faskan another speculation: since you are doing build in container the resulting binary might be of different os/architecture. The container probably produces Linux's ELF, but your mac expects Mach-O.

The error code 126 matches with this.

[faskan]

@faskan another speculation: since you are doing build in container the resulting binary might be of different os/architecture.

The container probably produces Linux's ELF, but your mac expects Mach-O.

@matejvasek yes, it happens only with container build flag. And you are right the build produces a Linux binary. But since I'm running it with Dquarkus.native.container-build=true I'm expecting the whole build process (including tests) to run in a container. But somehow quarkus builds the binary using a container and executes the tests in host OS.

[matejvasek]

You probably will need to install GraalVM locally.

[matejvasek]

I think it just do build in container and the result binary is copied outside the container to host, and tests are run locally then.

[matejvasek]

I'm expecting the whole build process (including tests) to run in a container.

That would be probably feature request.

[matejvasek]

cc @iocanel @geoand you since I see you in the history of container-image extensions.

[geoand]

Let me try and understand this.

You are building the native binary using -Dquarkus.native.container-build=true on a MacOS and then trying to launch that binary in the tests? That won't work. You'll need to build a container for that native binary using one of our container image extensions

[matejvasek]

then trying to launch that binary in the tests?

@geoand I think the issues is the the test is being run automatically.

[matejvasek]

So it is not immediately clear what is happening.

[faskan]

yes, when you do a quarkus build, the tests run automatically. So for Mac users, either they have to ignore the native tests or install graalvm locally and not use the container build flag.

[matejvasek]

@geoand I know it's logical consequence that test fail, but at least could it fail with better message? For example: Bad Exec Format.

[geoand]

Yeah, that makes sense

[geoand]

https://github.com/quarkusio/quarkus/blob/2.9.0.Final/test-framework/common/src/main/java/io/quarkus/test/common/DefaultNativeImageLauncher.java#L112 is where the change would need to go for anyone interested

[matejvasek]

@geoand what about:

diff --git a/test-framework/common/src/main/java/io/quarkus/test/common/LauncherUtil.java b/test-framework/common/src/main/java/io/quarkus/test/common/LauncherUtil.java
index b425ad8c60..94a12c7e58 100644
--- a/test-framework/common/src/main/java/io/quarkus/test/common/LauncherUtil.java
+++ b/test-framework/common/src/main/java/io/quarkus/test/common/LauncherUtil.java
@@ -10,6 +10,7 @@ import java.nio.file.Path;
 import java.time.Duration;
 import java.util.ArrayList;
 import java.util.List;
+import java.util.Locale;
 import java.util.ServiceLoader;
 import java.util.concurrent.CountDownLatch;
 import java.util.concurrent.TimeUnit;
@@ -94,8 +95,18 @@ public final class LauncherUtil {
                     "Interrupted while waiting to determine the status of process '" + quarkusProcess.pid() + "'.");
         }
         if (!quarkusProcess.isAlive()) {
-            throw new RuntimeException("Unable to successfully launch process '" + quarkusProcess.pid() + "'. Exit code is: '"
-                    + quarkusProcess.exitValue() + "'.");
+            String message = "Unable to successfully launch process '" + quarkusProcess.pid() + "'. Exit code is: '"
+                    + quarkusProcess.exitValue() + "'.";
+
+            String osName = System.getProperty("os.name", "generic").toLowerCase(Locale.ROOT);
+            boolean isUnix = osName.contains("mac") || osName.contains("darwin") || osName.contains("linux");
+            boolean isWin = osName.contains("windows");
+            int exit = quarkusProcess.exitValue();
+            if ((isUnix && exit == 126) || (isWin && exit == 9009)) {
+                message += System.lineSeparator()
+                        + "Hint: this may be caused by building the native application in a container while the host OS or architecture differs.";
+            }
+            throw new RuntimeException(message);
         }
     }

The output then is:

[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Errors: 
[ERROR]   NativeFunctionIT.testFunctionIntegration » Runtime Unable to successfully launch process '393167'. Exit code is: '126'.
Hint: this may be caused by building the native application in a container while the host OS or architecture differs.
[INFO] 
[ERROR] Tests run: 2, Failures: 0, Errors: 1, Skipped: 1
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  7.170 s
[INFO] Finished at: 2022-05-12T17:39:03+02:00
[INFO] ------------------------------------------------------------------------

[geoand]

That works too

[matejvasek]

@faskan would be the message suggested above sufficient to help user?

[faskan]

@matejvasek the message will definitely help a user to understand the problem. But I am not sure if this would be a good user experience.

I think the documentation https://quarkus.io/guides/building-native-image#container-runtime should also be corrected accordingly. It should simply say that "quarkus build --native -Dquarkus.native.container-build=true" works only on a Linux machine. For Mac or other OS, the native tests has to be disabled.

By default, native tests are enabled for a starter project and if any newbie (who doesn't want to install graalvm locally) is going to try out this way and the build fails even without making a single code change.