search

grpc / grpc

The C based gRPC (C++, Python, Ruby, Objective-C, PHP, C#)
https://grpc.io
Apache License 2.0
41.59k stars 10.5k forks source link

grpc-tools build missing for arm64 macOS #25755

Open bouk opened 3 years ago

bouk commented 3 years ago

It's not possible to use grpc-tools on M1 macs right now, because there's no binary distribution of the Ruby gRPC compiler being shipped. Something will have to be done on the internal artifact build side to make this happen.

jtattermusch commented 3 years ago

we currently don't provide support for the arm64 M1 macs, since we have no hardware to test on. For now, you need to use rosetta emulation to run on the new arm64 macs. We might add this support at some point, but there is no ETA.

bouk commented 3 years ago

@jtattermusch I made a PR so the x86_64 binary gets picked up at least: https://github.com/grpc/grpc/pull/25756

stale[bot] commented 3 years ago

This issue/PR has been automatically marked as stale because it has not had any update (including commits, comments, labels, milestones, etc) for 30 days. It will be closed automatically if no further update occurs in 7 day. Thank you for your contributions!

sobrinho commented 3 years ago

@jtattermusch what do you need to test it?

I'm sure the community can help, I have a M1 and I could help if you provide the instructions :)

jtattermusch commented 3 years ago

@jtattermusch what do you need to test it?

I'm sure the community can help, I have a M1 and I could help if you provide the instructions :)

Thanks, but it's not about running a one-time test and be done with it. We need continous testing to be able to ensure things won't break in the future. So we need mac M1s integrated with our internal testing platform and that's far from trivial.

jtattermusch commented 3 years ago

related https://github.com/grpc/grpc/issues/26391

sobrinho commented 3 years ago

Maybe having instructions on how to compile from source would at least unblock to use grpc locally, until we have a definitive solution.

nicolasnoble commented 3 years ago

Also related: https://github.com/actions/virtual-environments/issues/2187

On Thu, Sep 16, 2021 at 3:38 PM Gabriel Sobrinho @.***> wrote:

Maybe having instructions on how to compile from source would at least unblock to use grpc locally, until we have a definitive solution.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/grpc/grpc/issues/25755#issuecomment-921305927, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABXRXJRBKPBSYZ5RVMXFRITUCJWXXANCNFSM4ZMGFKSA .

jtattermusch commented 3 years ago

Maybe having instructions on how to compile from source would at least unblock to use grpc locally, until we have a definitive solution.

The way the grpc-tools gem is built by our CI is here: https://github.com/grpc/grpc/blob/master/tools/run_tests/artifacts/build_package_ruby.sh

basically, you need to first build the protoc binaries and grpc_ruby_plugin for all supported platforms (this step is quite unrelated to the ruby build process, since you just need the binaries), then you put the binaries in the right location where gem build expects them and finally you simply run gem build grpc-tools.gemspec. So besides building the protoc / protoc plugin binaries, building the grpc-tools gems is trivial.

Building grpc-tools for your own needs locally is absolutely doable (but obviously the gems built that way won't get any kind of official support from the grpc team).

dweekly commented 2 years ago

Hey folks, really would love to have first-class grpc-tools support for M1 macs since we're moving over from Intel-based to M1-based systems for doing our development and lean heavily on a grpc-tools stack for our workflow. I'm pretty sure I'm not the only one in this boat, so pretty please do find a way to support arm64!

nicolasnoble commented 2 years ago

@dweekly please see https://github.com/actions/virtual-environments/issues/2187

dweekly commented 2 years ago

@nicolasnoble Thanks so much; the linked thread indicates that GitHub is thinking that M1 support in Actions may be available sometime in 2022. https://github.com/actions/virtual-environments/issues/2187#issuecomment-939033228

I can appreciate the team's desire for quality in wanting to incorporate M1 Actions as part of your release flow, but it's also a non-super experience for the buyer of a modern, popular developer computer to not be able to use a core piece of software because one trillion dollar company is waiting for another trillion dollar company to do something.

Since I'm definitely not going to be the last person to run across this, perhaps it would be possible to have some sort of error pop up for those who encounter this with a link and guidance? Or to offer support in "preview"/beta/hand-cranked until full M1 GitHub actions are available for the team?

I'm appreciative of the hard work your team does and trying to be empathetic to devs wanting to use the grpc tools.

sobrinho commented 2 years ago

Looks like 1.42.0.pre1 was released, even include a version for arm64-darwin but it doesn't work yet

$ bin/rails db:migrate
rails aborted!
LoadError: dlopen(/Users/sobrinho/.gem/ruby/2.7.2/gems/grpc-1.42.0.pre1-universal-darwin/src/ruby/lib/grpc/2.7/grpc_c.bundle, 9): no suitable image found.  Did find:
    /Users/sobrinho/.gem/ruby/2.7.2/gems/grpc-1.42.0.pre1-universal-darwin/src/ruby/lib/grpc/2.7/grpc_c.bundle: mach-o, but wrong architecture
    /Users/sobrinho/.gem/ruby/2.7.2/gems/grpc-1.42.0.pre1-universal-darwin/src/ruby/lib/grpc/2.7/grpc_c.bundle: mach-o, but wrong architecture - /Users/sobrinho/.gem/ruby/2.7.2/gems/grpc-1.42.0.pre1-universal-darwin/src/ruby/lib/grpc/2.7/grpc_c.bundle
/Users/sobrinho/.gem/ruby/2.7.2/gems/activesupport-6.1.3.2/lib/active_support/dependencies.rb:332:in `require'
/Users/sobrinho/.gem/ruby/2.7.2/gems/activesupport-6.1.3.2/lib/active_support/dependencies.rb:332:in `block in require'
/Users/sobrinho/.gem/ruby/2.7.2/gems/activesupport-6.1.3.2/lib/active_support/dependencies.rb:299:in `load_dependency'
/Users/sobrinho/.gem/ruby/2.7.2/gems/activesupport-6.1.3.2/lib/active_support/dependencies.rb:332:in `require'
/Users/sobrinho/.gem/ruby/2.7.2/gems/grpc-1.42.0.pre1-universal-darwin/src/ruby/lib/grpc/grpc.rb:20:in `<top (required)>'
/Users/sobrinho/.gem/ruby/2.7.2/gems/grpc-1.42.0.pre1-universal-darwin/src/ruby/lib/grpc.rb:19:in `require_relative'
/Users/sobrinho/.gem/ruby/2.7.2/gems/grpc-1.42.0.pre1-universal-darwin/src/ruby/lib/grpc.rb:19:in `<top (required)>'
sobrinho commented 2 years ago 
$ bundle update grpc
...
Installing grpc 1.42.0.pre1 (universal-darwin)
...

$ bin/rails db:migrate
rails aborted!
LoadError: dlopen(/Users/sobrinho/.gem/ruby/2.7.2/gems/grpc-1.42.0.pre1-universal-darwin/src/ruby/lib/grpc/2.7/grpc_c.bundle, 9): no suitable image found.  Did find:
    /Users/sobrinho/.gem/ruby/2.7.2/gems/grpc-1.42.0.pre1-universal-darwin/src/ruby/lib/grpc/2.7/grpc_c.bundle: mach-o, but wrong architecture
    /Users/sobrinho/.gem/ruby/2.7.2/gems/grpc-1.42.0.pre1-universal-darwin/src/ruby/lib/grpc/2.7/grpc_c.bundle: mach-o, but wrong architecture - /Users/sobrinho/.gem/ruby/2.7.2/gems/grpc-1.42.0.pre1-universal-darwin/src/ruby/lib/grpc/2.7/grpc_c.bundle

Although, if I do:

$ gem uninstall grpc
[uninstall all versions]

$ gem install grpc:1.42.0.pre1
Fetching grpc-1.42.0.pre1-arm64-darwin.gem
Successfully installed grpc-1.42.0.pre1-arm64-darwin
Parsing documentation for grpc-1.42.0.pre1-arm64-darwin
Installing ri documentation for grpc-1.42.0.pre1-arm64-darwin
Done installing documentation for grpc after 0 seconds
1 gem installed

It does install the correct architecture and then the app works.

henriquebremenkanp commented 2 years ago

@sobrinho I'm new to ruby and I'm not sure if it's the right solution, but maybe you need to add arm64-darwin-20 to your Gemfile.lock platforms. I think it's what makes bundler to get the arm64-darwin instead of the universal-darwin. Not sure but maybe worth trying.

Btw, congratulations to the team on the newest v1.42.0 release! I just noticed the latest gem is still released as 1.42.0.pre1

Does it only need a separate action to be triggered or is there anything left to be worked on?

strangeweb3guy commented 2 years ago

Any updates? Still see that issue node-pre-gyp ERR! stack Error: 404 status code downloading tarball https://node-precompiled-binaries.grpc.io/grpc-tools/v1.8.0/darwin-arm64.tar.gz

brodock commented 2 years ago

@kiskesis this is specific to ruby gem, I'm sure other language will also have the problem, but please find/open a different issue

github-usr-1 commented 2 years ago

basically, you need to first build the protoc binaries and grpc_ruby_plugin for all supported platforms

@jtattermusch Thanks for describing how to build grpc-tools.gemspec. Can you also provide information on how to build the protoc binaries and grpc_ruby_plugin?

jtattermusch commented 2 years ago

basically, you need to first build the protoc binaries and grpc_ruby_plugin for all supported platforms

@jtattermusch Thanks for describing how to build grpc-tools.gemspec. Can you also provide information on how to build the protoc binaries and grpc_ruby_plugin?

Isn't a bit complicated, but this is a good place to start: https://github.com/grpc/grpc/blob/3c558dd7962f2f2aea6b6a5759f65fa6f11db847/tools/run_tests/artifacts/artifact_targets.py#L413-L418

The problem is that our CI currently doesn't have access to ARM macs, so there is nothing the build those binaries on (and without being able to build those binaries on CI there isn't really much we can do here).

sobrinho commented 2 years ago

What about doing it manually when a new version is released?

It's tedious and error prone but better than not having it at all.

Maybe it can be compiled into a single script to build everything and on every release, we can do it ourselves and submit to a private repository or even provide the compiled version here for the owner to push to public rubygems.

aspin commented 2 years ago

In case this is helpful to anyone –– I do dev work from an M1 and the work around I ended up using was building everything from a Docker image. Something like this, built with the --platform linux/amd64 flag include while building for compatibility:

FROM node:lts

# grpc_tools_node_protoc_ts for typescript bindings
RUN npm install -g grpc-tools grpc_tools_node_protoc_ts
RUN mkdir -p /home/node/out
RUN mkdir -p /home/node/in

WORKDIR "/home/node/in"

Then it's just a matter of mounting volumes in/out:

    docker run -v $(CURDIR)/server/proto:/home/node/out \
               -v $(CURDIR)/proto:/home/node/in $(DOCKER_IMAGE) \
        grpc_tools_node_protoc \
            --plugin=protoc-gen-ts=/usr/local/bin/protoc-gen-ts \
            --ts_out=grpc_js:../out \
            --js_out=import_style=commonjs,binary:../out \
            --grpc_out=grpc_js:../out \
            api.proto
lloeki commented 1 year ago

from an M1 [...] Something like this, built with the --platform linux/amd64

This only works if the Linux VM where docker is setup has binfmt_misc set up to run foreign x86_64 binaries via either Rosetta 2 for Linux (which requires using Virtualization.framework) or QEMU (which is slow).

lloeki commented 1 year ago

(reposting my comment here for visibility)

I feel like it's more than that, the gem appears to break several rubygems expectations:

The gem is published as ruby (which means "pure ruby", or abusively "compatible with any platform because it builds native components on the spot") but is in fact thin packaging for target-specific binaries:

$ env GEM_HOME=fake gem install grpc-tools
1 gem installed

$ ls -1 fake/gems/grpc-tools-1.56.2 fake/gems/grpc-tools-1.56.2/bin

fake/gems/grpc-tools-1.56.2:
README.md
bin
platform_check.rb
version.rb

fake/gems/grpc-tools-1.56.2/bin:
grpc_tools_ruby_protoc
grpc_tools_ruby_protoc_plugin
x86-linux
x86-windows
x86_64-linux
x86_64-macos
x86_64-windows

None but the one it ultimately runs on would be executable (without binfmt_misc trickery + a compatible userland), and so are dead weight.

Each one of these should be published as individual binary platform gems:

For Windows I'm less certain, could be mswin or mingw thingies:

-https://github.com/rubygems/rubygems/blob/8af163866564c6a2afb9fd3cd51f452459dd9f06/lib/rubygems/platform.rb#L103-L127

And in addition, what this issue requests:

The ruby gem platform should be attempting to build binaries from source upon install, or worst case, be a shim.

In addition, this case .. when bit of code falls through to return x86 on aarch64-linux:

  def PLATFORM.architecture
    host_cpu = RbConfig::CONFIG['host_cpu']

    # When we're on arm in macOS, we can rely on Rosetta and use the x86_64 binary
    return 'x86_64' if RbConfig::CONFIG['host_os'] =~ /darwin/ && host_cpu =~ /arm|aarch/

    case host_cpu
      when /x86_64/
        'x86_64'
      else
        'x86'
    end
  end

The java gem platform should include all binaries because it cannot be known which platform it will run on. But then I suppose JRuby is not a current target because the code above fails for JRuby.

For reference, here are some implementations:

lloeki commented 1 year ago

Some additional notes:

† can be achieved by statically linking C++ stdlib in and taking care that no glibc-isms are used

tebeco commented 4 months ago

Yep that problem still persists with the upcoming M4

tebeco commented 3 months ago

we currently don't provide support for the arm64 M1 macs, since we have no hardware to test on. For now, you need to use rosetta emulation to run on the new arm64 macs. We might add this support at some point, but there is no ETA.

somehow, @lloeki / @jtattermusch someone has to review the idea of relying on Rosetta 2 Sonoma 14.5 broke stuff in that translation layer https://discord.com/channels/143867839282020352/1203416343932833853/1243514842091094089 https://forums.developer.apple.com/forums/thread/755840

TLDR: GRPC tool needs to be natively compliant with ARM processor, including Apple Silicon this is not a sustainable solution

tebeco commented 1 month ago

🦤 any news on google side for GRPC to be useable on ARM64 / Apple Silicon processor ?

Curious what's the google plan with mobile/desktop cpu moving to ARM64

also cheap cloud compute now runs on ARM64 it's honestly weird post M1 but we're now close to M4 and 2025. as a no ice in the field i fail to see any clear sign for support and things moving forward "actively"