Support for M1 Macs (osx-aarch_64)

cmardini commented 3 years ago

% java -jar .m2/repository/com/github/os72/protoc-jar/3.11.4/protoc-jar-3.11.4.jar -pp
------------------------------------------------------------------------
Detecting the operating system and CPU architecture
------------------------------------------------------------------------
os.detected.name: osx
os.detected.arch: aarch_64
os.detected.version: 11.1
os.detected.version.major: 11
os.detected.version.minor: 1
os.detected.classifier: osx-aarch_64
Detected platform: osx-aarch_64 (mac os x/aarch64)

Would it be possible to add to protoc-jar/bin// a protoc--osx-aarch_64.exe? Temporarily, it can probably even be a copy of protoc-*-osx-x86_64.exe as it should run on M1 macs.

I also have a brew installed protoc that seems to be compiled for arm, if you need it.

% file /opt/homebrew/Cellar/protobuf/3.14.0/bin/protoc
/opt/homebrew/Cellar/protobuf/3.14.0/bin/protoc: Mach-O 64-bit executable arm64

% ls -l /opt/homebrew/Cellar/protobuf/3.14.0/bin/protoc
-r-xr-xr-x  1 cmardini  staff  71872  4 Jan 16:22 /opt/homebrew/Cellar/protobuf/3.14.0/bin/protoc

% md5 /opt/homebrew/Cellar/protobuf/3.14.0/bin/protoc 
MD5 (/opt/homebrew/Cellar/protobuf/3.14.0/bin/protoc) = 0da061ae62c179f668ff4a544377cec3

The protoc-jar-maven-plugin seems to request this for the embedded protoc.

[INFO] Protoc version: 3.11.0
protoc-jar: protoc version: 3.11.0, detected platform: osx-aarch_64 (mac os x/aarch64)
protoc-jar: downloading: https://repo.maven.apache.org/maven2/com/google/protobuf/protoc/maven-metadata.xml
protoc-jar: saved: /var/folders/gq/883qwjg91_x4pyq995m7vct40000gn/T/protocjar.webcache/com/google/protobuf/protoc/maven-metadata.xml
protoc-jar: downloading: https://repo.maven.apache.org/maven2/com/github/os72/protoc/maven-metadata.xml
protoc-jar: saved: /var/folders/gq/883qwjg91_x4pyq995m7vct40000gn/T/protocjar.webcache/com/github/os72/protoc/maven-metadata.xml
protoc-jar: cached: /var/folders/gq/883qwjg91_x4pyq995m7vct40000gn/T/protocjar.webcache/com/google/protobuf/protoc/maven-metadata.xml
protoc-jar: downloading: https://repo.maven.apache.org/maven2/com/google/protobuf/protoc/3.11.0/protoc-3.11.0-osx-aarch_64.exe
protoc-jar: cached: /var/folders/gq/883qwjg91_x4pyq995m7vct40000gn/T/protocjar.webcache/com/github/os72/protoc/maven-metadata.xml
protoc-jar: downloading: https://repo.maven.apache.org/maven2/com/github/os72/protoc/3.11.0/protoc-3.11.0-osx-aarch_64.exe
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.323 s
[INFO] Finished at: 2021-01-05T09:25:11-05:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.github.os72:protoc-jar-maven-plugin:3.11.4:run (default) on project webapi-grpc-clients: Error extracting protoc for version 3.11.0: Unsupported platform: protoc-3.11.0-osx-aarch_64.exe -> [Help 1]

To workaround, I've done the following:

mvn install:install-file -DgroupId=com -DartifactId=github -Dversion=os72 -Dclassifier=3.14.0 -Dpackaging=protoc -Dfile=/opt/homebrew/Cellar/protobuf/3.14.0/bin/protoc

and then in the plugin configuration in pom.xml:

                        <protocVersion>3.14.0</protocVersion>
                        <protocArtifact>com:github:os72:protoc:3.14.0</protocArtifact>

os72 commented 3 years ago

Ack. I see. We're saying special handling for osx-aarch_64:

look for native osx-aarch_64 binary
if not found, fallback to osx-x86_64 (cause M1 platform handles compatibility)

Correct?

If you could provide clean native binaries (static build, stripped, etc), good for wide consumption, I could host them on maven central: https://repo.maven.apache.org/maven2/com/github/os72/protoc/

You could make them available via PR: https://github.com/os72/protoc-jar/tree/master/bin

That would be a nice contribution. And if you're using the project professionally, I'd appreciate support (sponsorship) for it..

cmardini commented 3 years ago

I think if we populate the osx-aarch_64 binary, that would be sufficient.

I've built protoc with static linkage (./configure --disable-shared) and stripped the symbols. I'll make a PR shortly.

cmardini commented 3 years ago

Added a PR: https://github.com/os72/protoc-jar/pull/94

wangjia184 commented 3 years ago

when can this be released?

hazmeister commented 3 years ago

Suggested workaround for the time being:

Use brew to install the protobuf cli locally brew install protobuf https://formulae.brew.sh/formula/protobuf
Set the protocCommand property to protoc

I was able to add the property to a m1 maven build profile

<properties>
    <protocCommand>protoc</protocCommand>
</properties>

nikitadol commented 3 years ago

protoc-3.17.3-osx-aarch_64.exe https://repo1.maven.org/maven2/com/google/protobuf/protoc/3.17.3/

vborbely commented 3 years ago

protoc-3.17.3-osx-aarch_64.exe https://repo1.maven.org/maven2/com/google/protobuf/protoc/3.17.3/

I created this comment for the solution.

https://github.com/protocolbuffers/protobuf/issues/8062#issuecomment-892689597

cmardini commented 3 years ago

As per @hazmeister's comment, I added the following profile to my pom.xml that uses protoc. Maybe it can be shortened.


  <profile>
    <build>
      <plugins>
        <plugin>
          <groupId>com.github.os72</groupId>
          <artifactId>protoc-jar-maven-plugin</artifactId>
          <executions> 
            <execution>
              <configuration>
                <protocCommand>protoc</protocCommand> <!-- brew install protobuf -->
              </configuration>
            </execution> 
          </executions>
        </plugin>
      </plugins>
    </build>
    <activation>
      <os>
        <name>mac os x</name>
        <arch>aarch64</arch>
        <family>mac</family>
      </os>
    </activation>
  </profile>
</profiles>

os72 commented 3 years ago

I'm afraid the pace in this project will remain slow until more support/sponsorship can be found (most users are enterprises)..

Rigoberto-Vazquez commented 2 years ago

in the meantime, I was able to run the process in a Apple M1 chip using the parameter "-Dos.arch=x86_64" with the maven goals

pratikraj1 commented 2 years ago

Can someone provide final step to how to resolve this on M1 mac ?

slachiewicz commented 2 years ago

simply do something like


<plugin>
        <groupId>com.github.os72</groupId>
        <artifactId>protoc-jar-maven-plugin</artifactId>
        <version>3.11.4</version>
       <configuration>
         <protocArtifact>com.google.protobuf:protoc:3.21.1</protocArtifact>

BLACKCAT25896 commented 2 years ago

Where I can add those codes or suggestion codes? `

protoc

`

os72 commented 2 years ago

Where I can add those codes or suggestion codes? <properties> <protocCommand>protoc</protocCommand> </properties>

"properties" doesn't seem right, it goes under <configuration>. example: https://github.com/os72/protoc-jar/issues/93#issuecomment-893491180

slee500 commented 2 years ago

in the meantime, I was able to run the process in a Apple M1 chip using the parameter "-Dos.arch=x86_64" with the maven goals

@Rigoberto-Vazquez thanks for this -- works for me too.

Another way I found that works is to switch to using a x86_64 JDK instead. Seems to run just fine on my M1 Mac.

Reason94 commented 2 years ago

I found that works. Add the following configuration to maven settings.xml.

<profile>
      <id>apple-m1</id>
      <properties>
        <os.detected.classifier>osx-x86_64</os.detected.classifier>
      </properties>
</profile>

Hwloser commented 1 year ago

firstly, clone protobuf sourc code: git@github.com:protocolbuffers/protobuf.git
check out your specify version: (e.g. v3.11.4: git checkout v3.11.4 -b v3.11.4)
compile profobuf of specify version a.(optional) ( in version v3.11.4, that must to generate configure.sh): autoconf b.(optional) (then generate Makefile): ./configure c. make -j $(nproc)
1. then you will find protoc under src directory
2. change the configuration <-- protoc-jar-maven-plugin --> a. add protocCommand tag b. example:
```
<configuration>
  <protocCommand>/Users/huanwei/git_files/github/protobuf/src/protoc</protocCommand>
  <protocVersion>${protoc.version}</protocVersion>
  <inputDirectories>
        <include>src/main/resources</include>
  </inputDirectories>
</configuration>
```
then re-run maven command, you will success

ennerf commented 1 year ago

Don't you only need to specify a <protocVersion> of 3.17.3 or higher?

I'm not using protoc-jar by itself, but at least for the protoc-jar-maven-plugin that is sufficient.

lassounski commented 1 year ago

Make sure that you're running on a JDK that is x86_64.

I was previously running on Zulu JDK with arch64 architecture and having that issue.

After switching to OpenJDK the problem was gone.

YuxiangZhuang commented 1 year ago

Make sure that you're running on a JDK that is x86_64.

I was previously running on Zulu JDK with arch64 architecture and having that issue.

After switching to OpenJDK the problem was gone.

it's good for me

ClaudioJrCode commented 1 month ago

echo "protoc_platform=osx-x86_64" >> ~/.gradle/gradle.properties

vsop-479 commented 2 weeks ago

As per @hazmeister's comment, I added the following profile to my pom.xml that uses protoc. Maybe it can be shortened.

Very nice, It works for me.

os72 / protoc-jar

Support for M1 Macs (osx-aarch_64) #93