search

apache / maven-mvnd

Apache Maven Daemon
https://maven.apache.org/
Apache License 2.0
2.9k stars 207 forks source link

Cannot use mvnd due to missing CPU features (on an older machine) #1029

Closed hhoffstaette closed 3 months ago

hhoffstaette commented 3 months ago

Thank you very much for mvnd! I'm running the prebuilt binary just fine on my AMD Zen2 laptop, but when I tried the same on an older machine, I only got a sad message about a target architecture mismatch:

The current machine does not support all of the following CPU features that are required by the image: [CX8, CMOV, FXSR, MMX, SSE, SSE2, SSE3, SSSE3, SSE4_1, SSE4_2, POPCNT, LZCNT, AVX, AVX2, BMI1, BMI2, FMA].
Please rebuild the executable with an appropriate setting of the -march option.

In my case the CPU (a SandyBridge box that still works perfectly fine) is missing LZCNT, AVX2, BMI1/2, FMA.

I know this is an old system, but it still works fine and in order to reach the widest audience it would IMHO be great if future releases could be built with a more conservative -march target. mvnd hardly benefits from BMI/FMA and probably only in a few synthetic benchmarks from AVX2, so this would not have any measurable performance impact.

cstamas commented 3 months ago

I don't think mvnd directly "asks" for CPU features, this sounds more like GraalVM issue IMHO (that produces native binary out from Java source), really unsure.

mthmulders commented 3 months ago

I remember having seen a similar issue in one of my pet projects. Adding -march=compatibility to the build flags fixed it. See native-image build options for reference.

gnodet commented 3 months ago

As a workaround, one can export the env variableMVND_CLIENT=jvm to bypass the native client.

TobiX commented 3 months ago

It seems like GraalVM defaults to x86-64-v3, which (playing devil's advocate here), seems reasonable to me (According to Wikipedia this covers CPUs released in the last 10 years, starting with Intel Haswell).

SandyBridge (released ~2011) falls right outside that range...

If one really wants to cater to CPUs that old, I would suggest -march=x86-64-v2, since I suspect compatibility will drop us right back to the x86_64 baseline from 2003...

cstamas commented 3 months ago

Tried several -march options on Fedora 40 with

openjdk version "22.0.1" 2024-04-16
OpenJDK Runtime Environment GraalVM CE 22.0.1+8.1 (build 22.0.1+8-jvmci-b01)
OpenJDK 64-Bit Server VM GraalVM CE 22.0.1+8.1 (build 22.0.1+8-jvmci-b01, mixed mode, sharing)

https://gist.github.com/cstamas/2d5d3cb45d60bdaf72730a1620d3f1c5

Tool used https://github.com/pkgw/elfx86exts/releases/tag/elfx86exts%400.6.2

hhoffstaette commented 3 months ago

Tried several -march options on Fedora 40 with

That doesn't look right..neither compatibility nor v1/v2 should use e.g. AVX2 or AVX512. But thanks for taking a look.

cstamas commented 3 months ago

To reproduce, use mvnd-1.x branch and I had these changes (yes, I also enabled FFM, so it may affected it):

diff --git a/client/pom.xml b/client/pom.xml
index ba943a5a..687ace27 100644
--- a/client/pom.xml
+++ b/client/pom.xml
@@ -223,11 +223,12 @@
               <skip>false</skip>
               <mainClass>org.mvndaemon.mvnd.client.DefaultClient</mainClass>
               <imageName>mvnd</imageName>
-              <buildArgs>--no-server
-                         --no-fallback
-                         --allow-incomplete-classpath
+              <buildArgs>--no-fallback
+                         -H:+UnlockExperimentalVMOptions
                          ${graalvm-native-static-opt}
                          ${graalvm-native-glibc-opt}
+                         -march=native
+                         -H:+ForeignAPISupport
                          -H:IncludeResources=org/mvndaemon/mvnd/.*
                          -H:IncludeResources=mvnd-bash-completion.bash
                          -H:-ParseRuntimeOptions

Then mvn clean package -P native -f client/ and just did change march between:

Also, FTR my CPU is i7-13700.

asamolov commented 3 months ago

Same issue on more or less modern MacBook Pro M1 with mvnd 1.0.0 installed with sdkman.

➜  ~ sdk use mvnd 1.0.0

Using mvnd version 1.0.0 in this shell.
➜  ~ mvnd -v
The current machine does not support all of the following CPU features that are required by the image: [CX8, CMOV, FXSR, MMX, SSE, SSE2, SSE3, SSSE3, SSE4_1, SSE4_2, POPCNT, LZCNT, AVX, AVX2, BMI1, BMI2, FMA].
Please rebuild the executable with an appropriate setting of the -march option.%

Woks fine with previously available version 1.0-m8-m40.

➜  ~ sdk use mvnd 1.0-m8-m40

Using mvnd version 1.0-m8-m40 in this shell.
➜  ~ mvnd -v
Unable to find the root directory. Create a .mvn directory in the root directory or add the root="true" attribute on the root project's model to identify it.
Apache Maven Daemon (mvnd) 1.0-m8 darwin-amd64 native client (0f4bdb6df5e74453d8d558d292789da4e66a7933)

Machine spec:

➜  ~ system_profiler SPSoftwareDataType SPHardwareDataType
...
    Hardware Overview:

      Model Name: MacBook Pro
      Model Identifier: MacBookPro18,2
      Chip: Apple M1 Max
      Total Number of Cores: 10 (8 performance and 2 efficiency)
      Memory: 64 GB
TobiX commented 3 months ago

@asamolov Why does sdkman install the x86_64 variant on arm64 hardware? Sounds a bit strange to run mvnd through Rosetta 2... It seems Rosetta implements a very "low" x86_64 compatibility level...

(which suggest this is not the same issue at all)

cstamas commented 3 months ago

For me this is what GraalVM 22 lists

$ native-image -march=list
On AMD64, the following machine types are available:

'compatibility'
  CPU features: all of 'x86-64'
'haswell'
  CPU features: all of 'x86-64' + SSE3 + SSSE3 + SSE4_1 + SSE4_2 + POPCNT + LZCNT + AVX + AVX2 + AES + CLMUL + BMI1 + BMI2 + FMA + F16C
'native'
  CPU features: CX8 + CMOV + FXSR + MMX + AMD_3DNOW_PREFETCH + SSE + SSE2 + SSE3 + SSSE3 + SSE4_1 + SSE4_2 + POPCNT + LZCNT + TSC + TSCINV_BIT + AVX + AVX2 + AES + ERMS + CLMUL + BMI1 + BMI2 + ADX + SHA + FMA + VZEROUPPER + FLUSH + FLUSHOPT + CLWB + SERIALIZE + RDTSCP + RDPID + FSRM + F16C + PKU + OSPKE + CET_IBT + CET_SS
'skylake'
  CPU features: all of 'haswell' + AMD_3DNOW_PREFETCH + ADX + FLUSHOPT
'skylake-avx512'
  CPU features: all of 'skylake' + AVX512F + AVX512DQ + AVX512CD + AVX512BW + AVX512VL + CLWB + PKU
'x86-64'
  CPU features: CX8 + CMOV + FXSR + MMX + SSE + SSE2
'x86-64-v1'
  CPU features: all of 'x86-64'
'x86-64-v2'
  CPU features: all of 'x86-64-v1' + SSE3 + SSSE3 + SSE4_1 + SSE4_2 + POPCNT
'x86-64-v3'
  CPU features: all of 'x86-64-v2' + LZCNT + AVX + AVX2 + BMI1 + BMI2 + FMA + F16C
'x86-64-v4'
  CPU features: all of 'x86-64-v3' + AVX512F + AVX512DQ + AVX512CD + AVX512BW + AVX512VL
asamolov commented 3 months ago

@TobiX thanks for the hint, let me check why it loads darwin-amd64 instead of darwin-aarch64.

UPD: indeed, I had an old sdkman version that loaded darwin-amd64 on aarch64. sdk selfupdate and re-installing mvnd solved the problem. Thanks again!

cstamas commented 3 months ago

Please retest with binaries built by CI from here: https://github.com/apache/maven-mvnd/actions/runs/9597334445

hhoffstaette commented 3 months ago

I tested the binary from the CI run and it works. 🙌 x86-64-v2 would have been fine too but whatever; thank you very much! ❤️

kwin commented 3 months ago

@cstamas Have you done some performance comparisons between native and compatibility on a modern CPU?

cstamas commented 3 months ago

Nope. @gnodet you?

kwin commented 3 months ago

I just fear that 99% of users have to suffer from that performance penalty for those 1% running on a very old CPU

gnodet commented 3 months ago

I don’t expect much penalty really. The client is mostly sitting waiting for messages on a socket and printing them to stdout. Compared to the build, I would assume the loss will be negligible.

Guillaume Nodet

Red Hat, Open Source Integration

Email: @.*** Web: http://fusesource.com Blog: http://gnodet.blogspot.com/

Le jeu. 20 juin 2024 à 19:14, Konrad Windszus @.***> a écrit :

I just fear that 99% of users have to suffer from that performance penalty for those 1% running on a very old CPU

— Reply to this email directly, view it on GitHub https://github.com/apache/maven-mvnd/issues/1029#issuecomment-2181170907, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAUQNQKBOX3BAK3FSAR3KLZIMEWTAVCNFSM6AAAAABJOMSLG6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBRGE3TAOJQG4 . You are receiving this because you were mentioned.Message ID: @.***>

cstamas commented 3 months ago

It may be better documented (PRs accepted), but it is hinted here: https://github.com/apache/maven-mvnd/blob/master/README.adoc#introduction

The mvnd (or mvnd.exe) is just a tiny native binary, that is "messenger" to background running daemons (that ARE Java). It is not "whole Maven" natively compiled, just this tiny binary. And all it does is passes user input to (selected) daemon and back, sends to console messages got from (selected) daemon.