apache / datafusion-comet

Apache DataFusion Comet Spark Accelerator
https://datafusion.apache.org/comet
Apache License 2.0
612 stars 113 forks source link

Bad CPU type in executable protoc-jar #227

Closed andygrove closed 3 weeks ago

andygrove commented 2 months ago

Describe the bug

I am trying to build Comet on a new Macbook Pro M3 Max and I am running into this error:

Cannot run program "/Users/andy/protoc5660975561245775818.exe": error=86, Bad CPU type in executable

Steps to reproduce

I installed openjdk@17 and protoc using homebrew. I am using the following command to build the project.

make release PROFILES="-Pspark-3.4"

Here is some of the log output:

Downloaded from central: https://repo.maven.apache.org/maven2/com/google/protobuf/protoc/3.19.6/protoc-3.19.6-osx-aarch_64.exe (12 MB at 1.6 MB/s)
protoc-jar: executing: [/var/folders/vv/fmb1n2hx3yqdmxbrv7shzyvr0000gn/T/protoc1458461729440081482.exe, --version]
protoc-jar: caught exception, retrying: java.io.IOException: Cannot run program "/var/folders/vv/fmb1n2hx3yqdmxbrv7shzyvr0000gn/T/protoc1458461729440081482.exe": error=86, Bad CPU type in executable

Expected behavior

No response

Additional context

No response

viirya commented 2 months ago

Hmm, I'm also using M3 Max, but I don't see the issue.

java --version
openjdk 11.0.22 2024-01-16
OpenJDK Runtime Environment Homebrew (build 11.0.22+0)
OpenJDK 64-Bit Server VM Homebrew (build 11.0.22+0, mixed mode)

Is it possible that you installed x86 JDK?

And, from the error message,

Downloaded from central: https://repo.maven.apache.org/maven2/com/google/protobuf/protoc/3.19.6/protoc-3.19.6-osx-aarch_64.exe (12 MB at 1.6 MB/s)
protoc-jar: executing: [/var/folders/vv/fmb1n2hx3yqdmxbrv7shzyvr0000gn/T/protoc1458461729440081482.exe, --version]
protoc-jar: caught exception, retrying: java.io.IOException: Cannot run program "/var/folders/vv/fmb1n2hx3yqdmxbrv7shzyvr0000gn/T/protoc1458461729440081482.exe": error=86, Bad CPU type in executable

Looks like it tries to run x86 protoc?

viirya commented 2 months ago

And do you have Xcode installed?

andygrove commented 2 months ago

I do have xcode installed. I tried with openjdk 11 as well.

% java --version 
openjdk 11.0.22 2024-01-16
OpenJDK Runtime Environment Homebrew (build 11.0.22+0)
OpenJDK 64-Bit Server VM Homebrew (build 11.0.22+0, mixed mode)

It seems to be arm64 build:

% file /opt/homebrew/Cellar/openjdk@11/11.0.22/bin/javac
/opt/homebrew/Cellar/openjdk@11/11.0.22/bin/javac: Mach-O 64-bit executable arm64

I also have protoc installed and seems to be arm64 version:

 % file /opt/homebrew/bin/protoc
/opt/homebrew/bin/protoc: Mach-O 64-bit executable arm64

I still see the same issue 😕

viirya commented 2 months ago

Is it possible due to this https://github.com/protocolbuffers/protobuf/pull/8557?

viirya commented 2 months ago

Do you install protobuf using Homebrew? This version protoc should be used instead of the downloaded one "/var/folders/vv/fmb1n2hx3yqdmxbrv7shzyvr0000gn/T/protoc1458461729440081482.exe".

andygrove commented 2 months ago

I tried installing protobuf with homebrew as well as manually. In either case, maven runs the downloaded one, which is not really an arm64 build even though it pretends to be.

As a workaround, I ran the following comment to enable x86 compatibility mode (as suggested in https://github.com/protocolbuffers/protobuf/pull/8557) and the build now works.

softwareupdate --install-rosetta
viirya commented 2 months ago

Hmm, it's weird. I don't remember I've installed rosetta. @sunchao Do you have it?

sunchao commented 2 months ago

I don't think I installed Rosetta either.

andygrove commented 3 weeks ago

This is no longer an issue for me and we have not had other reports of this happening, so will close this