protocolbuffers / protobuf

Protocol Buffers - Google's data interchange format
http://protobuf.dev
Other
64.53k stars 15.36k forks source link

Ruby google-protobuf-4.26.1-aarch64-linux precompiled gem is not compatible with linux-musl #16853

Open ntkme opened 1 month ago

ntkme commented 1 month ago

What version of protobuf and what language are you using? Version: 4.26.1 Language: Ruby

What operating system (Linux, Windows, ...) and version? Alpine Linux 3.19.1

What runtime / compiler are you using (e.g., python version or gcc version) ruby 3.3.1 (2024-04-23 revision c56cd86388) [aarch64-linux-musl]

What did you do? Steps to reproduce the behavior:

/ # gem install google-protobuf
Successfully installed google-protobuf-4.26.1-aarch64-linux
1 gem installed
/ # ruby -e "require 'google/protobuf'"
<internal:/usr/local/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:136:in `require': cannot load such file -- google/protobuf_c (LoadError)
    from <internal:/usr/local/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:136:in `require'
    from /usr/local/bundle/gems/google-protobuf-4.26.1-aarch64-linux/lib/google/protobuf_native.rb:15:in `rescue in <top (required)>'
    from /usr/local/bundle/gems/google-protobuf-4.26.1-aarch64-linux/lib/google/protobuf_native.rb:12:in `<top (required)>'
    from <internal:/usr/local/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:136:in `require'
    from <internal:/usr/local/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:136:in `require'
    from /usr/local/bundle/gems/google-protobuf-4.26.1-aarch64-linux/lib/google/protobuf.rb:57:in `<module:Protobuf>'
    from /usr/local/bundle/gems/google-protobuf-4.26.1-aarch64-linux/lib/google/protobuf.rb:15:in `<module:Google>'
    from /usr/local/bundle/gems/google-protobuf-4.26.1-aarch64-linux/lib/google/protobuf.rb:14:in `<top (required)>'
    from <internal:/usr/local/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:141:in `require'
    from <internal:/usr/local/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:141:in `rescue in require'
    from <internal:/usr/local/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:135:in `require'
    from -e:1:in `<main>'
<internal:/usr/local/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:136:in `require': Error loading shared library ld-linux-aarch64.so.1: No such file or directory (needed by /usr/local/bundle/gems/google-protobuf-4.26.1-aarch64-linux/lib/google/3.3/protobuf_c.so) - /usr/local/bundle/gems/google-protobuf-4.26.1-aarch64-linux/lib/google/3.3/protobuf_c.so (LoadError)
    from <internal:/usr/local/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:136:in `require'
    from /usr/local/bundle/gems/google-protobuf-4.26.1-aarch64-linux/lib/google/protobuf_native.rb:13:in `<top (required)>'
    from <internal:/usr/local/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:136:in `require'
    from <internal:/usr/local/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:136:in `require'
    from /usr/local/bundle/gems/google-protobuf-4.26.1-aarch64-linux/lib/google/protobuf.rb:57:in `<module:Protobuf>'
    from /usr/local/bundle/gems/google-protobuf-4.26.1-aarch64-linux/lib/google/protobuf.rb:15:in `<module:Google>'
    from /usr/local/bundle/gems/google-protobuf-4.26.1-aarch64-linux/lib/google/protobuf.rb:14:in `<top (required)>'
    from <internal:/usr/local/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:141:in `require'
    from <internal:/usr/local/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:141:in `rescue in require'
    from <internal:/usr/local/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:135:in `require'
    from -e:1:in `<main>'
<internal:/usr/local/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:136:in `require': cannot load such file -- google/protobuf (LoadError)
    from <internal:/usr/local/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:136:in `require'
    from -e:1:in `<main>'

What did you expect to see google-protobuf should work on aarch64-linux-musl just like how it works on x86_64-linux-musl

What did you see instead? google-protobuf prebuilt library is strictly linked to glibc and it's not compatible with musl-libc

Anything else we should know about your project / environment Please see rake-compiler's documentation regarding recent changes in rubygems in regarding to -linux, -linux-gnu, and -linux-musl platforms: https://github.com/rake-compiler/rake-compiler-dock?tab=readme-ov-file#linux-gnu-and-musl-important-details

ntkme commented 1 month ago

As a workaround, uninstalling the prebuilt gem and reinstall platform ruby to build from source fixes the issue:

/ # gem uninstall google-protobuf
Successfully uninstalled google-protobuf-4.26.1-aarch64-linux
/ # apk add alpine-sdk
...
/ # gem install --platform ruby google-protobuf
Fetching google-protobuf-4.26.1.gem
Building native extensions. This could take a while...
Successfully installed google-protobuf-4.26.1
1 gem installed
/ # ruby -e "require 'google/protobuf'"
ntkme commented 1 month ago

There are a few options here:

  1. Do not ship aarch64-linux gem, and let both aarch64-linux-gnu and aarch64-linux-musl platforms fallback to platform ruby and compile from source.
  2. Ship the current flavor, but label it as aarch64-linux-gnu, so that it does not get installed incorrectly on aarch64-linux-musl.
  3. Upgrade rake-compiler, and ship separate aarch64-linux-gnu and aarch64-linux-musl gems.

Again, please refer to rake-compiler's documentation on details: https://github.com/rake-compiler/rake-compiler-dock?tab=readme-ov-file#linux-gnu-and-musl-important-details

ntkme commented 3 weeks ago

@acozzette Can you please have a look when you get a chance?