Add ARM architecture support to Maven repository

bootjp commented 2 years ago

Motivation

Recently, computers with ARM CPU such as AWS Graviton, Apple M1 and Raspberry Pi are increasing. By releasing binaries to Maven that run on ARM CPUs, you can run them in a wide range of environments.

Desired solution

Adding Binary Support for ARM CPUs to the Maven Repository

Considered alternatives

Additional context

When I built rsocket-rpc-protobuf version 0.3.0 as a dependency on an M1 computer with ARM, I got the following error:.

Could not find rsocket-rpc-protobuf-0.3.0-osx-aarch_64.exe (io.rsocket.rpc:rsocket-rpc-protobuf:0.3.0).
     Searched in the following locations:
         https://repo.maven.apache.org/maven2/io/rsocket/rpc/rsocket-rpc-protobuf/0.3.0/rsocket-rpc-protobuf-0.3.0-osx-aarch_64.exe

mgodave commented 2 years ago

This is currently a pretty big blocker for me. I would love to help make some progress here. I haven't spent much time with C++ in the last decade, if someone has some pointers I'd be happy to help. I've been fiddling around with things for a bit and seem to currently be stuck on the fact that the most recent libprotoc headers have drifted. Is there any idea which version of the protonf compiler headers this was originally built with?

mgodave commented 2 years ago

Steps I took to unblock myself:

Download Protobuf 3.7.1 if you don't already have it
./configure --prefix=$prefix && make install song and dance. This will generate local ARM binaries and leave you with the appropriate include files. I have noticed that there is significant drift between two years ago (3.7.1) and now (3.17 ish) but haven't dug deeper to see where and what it would take to upgrade.

Index: rsocket-rpc-protobuf/build.gradle
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/rsocket-rpc-protobuf/build.gradle b/rsocket-rpc-protobuf/build.gradle
--- a/rsocket-rpc-protobuf/build.gradle (revision c3c47723ad85af9c7ba2edbf27494c7f1835a893)
+++ b/rsocket-rpc-protobuf/build.gradle (date 1652884125615)
@@ -48,7 +48,9 @@
     gcc(Gcc) {
       target("ppcle_64")
     }
-    clang(Clang) {}
+    clang(Clang) {
+      target("osx_aarch64")
+    }
   }

   platforms {
@@ -61,11 +63,14 @@
     ppcle_64 {
       architecture "ppcle_64"
     }
+    osx_aarch64 {
+      architecture "aarch64"
+    }
   }

   components {
     java_plugin(NativeExecutableSpec) {
-      if (arch in ['x86_32', 'x86_64', 'ppcle_64']) {
+      if (arch in ['x86_32', 'x86_64', 'ppcle_64', 'aarch64']) {
         // If arch is not within the defined platforms, we do not specify the
         // targetPlatform so that Gradle will choose what is appropriate.
         targetPlatform arch
@@ -78,11 +83,12 @@
     all {
       if (toolChain in Gcc || toolChain in Clang) {
         cppCompiler.define("RSOCKET_RPC_VERSION", version)
-        cppCompiler.args "--std=c++11"
+        cppCompiler.args "--std=c++11", "-I/home/mgodave/protobuf3.7.1/include/"
         addEnvArgs("CXXFLAGS", cppCompiler.args)
         addEnvArgs("CPPFLAGS", cppCompiler.args)
         if (osdetector.os == "osx") {
           cppCompiler.args "-mmacosx-version-min=10.7", "-stdlib=libc++"
+          linker.args "-L/home/mgodave/protobuf3.7.1/lib"
           addLibraryIfNotLinked('protoc', linker.args)
           addLibraryIfNotLinked('protobuf', linker.args)
         } else if (osdetector.os == "windows") {
Index: rsocket-rpc-core/build.gradle
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/rsocket-rpc-core/build.gradle b/rsocket-rpc-core/build.gradle
--- a/rsocket-rpc-core/build.gradle (revision c3c47723ad85af9c7ba2edbf27494c7f1835a893)
+++ b/rsocket-rpc-core/build.gradle (date 1652885729811)
@@ -48,7 +48,8 @@
     generatedFilesBaseDir = "${projectDir}/src/generated"

     protoc {
-        artifact = "com.google.protobuf:protoc"
+       // artifact = "com.google.protobuf:protoc"
+        path = "/home/protobuf3.7.1/bin/protoc"
     }
     plugins {
         rsocketRpc {

replace /home/mgodave/protobuf3.7.1 with your $prefix value
./gradlew build publishToMavenLocal

Make sure you have mavenLocal() in your repositories path for grade. You can now build your project on Mac m1 hardware...

This simply can unblock you. I still need to spend some time trying to put together a real patch so this project can begin publishing artifacts. One problem I already see is that the dependency chain with protoc does not have aarch64 binaries until 3.17.3 so we either need to do the full upgrade of prototbuf and the breaking header changes in the C++ or... well, we probably need to do that.

Anyway, hope this helps, I'm not using this library professionally, it's part of a side "for fun" project I'm working on so I may or may not have the time or motivation to do this work but perhaps this helps someone else get a head start if I don't do it myself.

mgodave commented 2 years ago

One problem I already see is that the dependency chain with protoc does not have aarch64 binaries until 3.17.3 so we either need to do the full upgrade of prototbuf and the breaking header changes in the C++ or... well, we probably need to do that.

Perhaps we can cross compile... It's been so long since I've done any of this in C++ land I would need to get some feedback or spend the time to explore what this would take, out may be easy to someone with more recent experience than myself.

mgodave commented 2 years ago

It looks like that diff is probably unnecessary, you can accomplish the same header and library include portions with command line options

https://github.com/rsocket/rsocket-rpc-java/blob/master/ci/travis.sh#L3

mgodave commented 2 years ago

This is s thread of discovery, feel free to follow along on my journey...

This may be a simple CI problem. I believe we may be able to add a CI target for an M1 platform and add the aarch64 target to the build file. I need to head into work but will try to put something together this evening for a PR.

mostroverkhov commented 2 years ago

the most recent libprotoc headers have drifted

@mgodave Original compiler plugin was coded /w protobuf 3.6.1 in 2018 and was not updated since that time.

Not sure update is worth the effort really - the library in its current form is likely an authentic "cruft". For starter, generated java code does not compile /w current RSocket-java.

Even if did there are metadata leaks and tracing problems, few other reported issues are rotting on this tracker.

Even if these are fixed there is RSocket/RSocket-java whose current GA release (1.1.2) performs plainly worse than GRPC and has 4 outstanding vulnerabilities (trivial server denial-of-service) that are mostly ignored for a year now.

rsocket / rsocket-rpc-java