grpc gem 1.3.4 LoadError: GLIBC_2.15 not found

[jacobvosmaer]

Please answer these questions before submitting your issue.

Should this be an issue in the gRPC issue tracker?

Yes.

What version of gRPC and what language are you using?

Ruby, grpc 1.3.4.

What operating system (Linux, Windows, …) and version?

Linux amd64. Two distros: Debian 7 and Centos 6.

What runtime / compiler are you using (e.g. python version or version of gcc)

Ruby 2.3.3

What did you do?

gem install grpc -v 1.3.4
ruby -r grpc -e 'puts :ok'

What did you expect to see?

ok

What did you see instead?

(On Debian 7)

$ ruby -r grpc -e 'puts :ok'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/grpc-1.3.4-x86_64-linux/src/ruby/lib/grpc/grpc.rb:35:in `require_relative': /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.15' not found (required by /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/grpc-1.3.4-x86_64-linux/src/ruby/lib/grpc/2.3/grpc_c.so) - /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/grpc-1.3.4-x86_64-linux/src/ruby/lib/grpc/2.3/grpc_c.so (LoadError)
    from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/grpc-1.3.4-x86_64-linux/src/ruby/lib/grpc/grpc.rb:35:in `<top (required)>'
    from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/grpc-1.3.4-x86_64-linux/src/ruby/lib/grpc/errors.rb:30:in `require_relative'
    from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/grpc-1.3.4-x86_64-linux/src/ruby/lib/grpc/errors.rb:30:in `<top (required)>'
    from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/grpc-1.3.4-x86_64-linux/src/ruby/lib/grpc.rb:32:in `require_relative'
    from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/grpc-1.3.4-x86_64-linux/src/ruby/lib/grpc.rb:32:in `<top (required)>'
    from /opt/gitlab/embedded/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:133:in `require'
    from /opt/gitlab/embedded/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:133:in `rescue in require'
    from /opt/gitlab/embedded/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:40:in `require'

Anything else we should know about your project / environment?

The same happens on Centos 6. Version 1.2.5 and earlier of the grpc gem did not have this problem.

Also see the exchange in https://github.com/google/protobuf/issues/2783#issuecomment-284577832 . There the resolution was to rebuild the gems using glibc 2.12. Edit: There the resolution was to rebuild the gems using glibc 2.12.

[apolcyn]

It does look like the GLIBC_2.15 dependency was newly added in the 1.3.4 grpc_c.so.

Not certain of the cause of the updated dependency on GLIBC_2.15 in grpc_c.so yet, but as a different workaround, is it feasible to upgrade libc6 on your system (I think a newer libc6 should provide the GLIBC_2.15)?

[jacobvosmaer]

is it feasible to upgrade libc6 on your system

Not really. We are shipping software for others to install; it does not run on 'our' system. Here is one recent Centos 6 package we shipped https://packages.gitlab.com/gitlab/gitlab-ce/packages/el/6/gitlab-ce-9.1.4-ce.0.el6.x86_64.rpm : 3k+ downloads

Debian 7 at the same version https://packages.gitlab.com/gitlab/gitlab-ce/packages/debian/wheezy/gitlab-ce_9.1.4-ce.0_amd64.deb 900+ downloads

These numbers exclude the paid version of GitLab and people who did not install that specific version. We are not in a position to tell thousands of administrators to update glibc, which seems to be a hard and unorthodox thing to do at least on Centos 6. https://stackoverflow.com/a/38317265/3665836

[jacobvosmaer]

@apolcyn is there really a hard dependency on glibc 2.15 in gRPC, or is that the version that was installed on the build machine for that gem release? In my experience it is usually the latter, i.e. an accidental dependency.

[brodock]

It does look like the GLIBC_2.15 dependency was newly added in the 1.3.4 grpc_c.so.

@apolcyn can we stick with semantic versioning and not bump hard dependencies in a patch number release?

[brodock]

(It sounded a little bit harsh my last comment, not the intention here, I'm just frustrated with this)

[apolcyn]

Indeed this breakage wasn't intentional, but bisecting this, it looks like out grpc_c.so native lib started depending on GLIBC_2.15 when a c-ares based resolver was added in (added between the 1.2 and 1.3 branches).

objdump -p on a grpc_c.so file starts showing:

Version References:
  ...
  required from libc.so.6:
    0x06969195 0x00 18 GLIBC_2.15
  ...

at that time.

Taking a look at what can do for this, but unfortunately this don't think this was due to a changed build environment.

[apolcyn]

cc @nicolasnoble @y-zeng do you have context on this? (the source appears to be in #7771)

[jacobvosmaer]

I think I just confirmed that grpc 1.3.4 installs without problems on Debian 7 (glibc 2.13) without problems. Edit: when not using the pre-built dynamic libraries

# Install RVM
gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
sudo apt-get install curl
\curl -sSL https://get.rvm.io | bash -s stable

# Install Ruby 2.3.4 (may need to start a new shell in between because RVM uses shell functions)
rvm install 2.3.4
rvm use 2.3.4
ruby -v

# Install grpc 1.3.4 without precompiled libraries
sudo apt-get install build-essential autoconf libtool
gem install grpc -v 1.3.4 --platform=ruby

# Test that the grpc gem loads
ruby -r grpc -e 'puts :ok'

Will check Centos 6 too.

[jacobvosmaer]

No luck for me with gem install grpc -v 1.3.4 --platform=ruby Centos 6 but that is for other reasons (boringssl error: stdalign.h: No such file or directory). I don't want to go down that rabbit hole right now.

@apolcyn I think that my post above answers my earlier question whether the dependency on glibc 2.15 is accidental or not: it must be accidental because I got grpc 1.3.4 to compile and load on Debian 7 with glibc 2.13.

[apolcyn]

To update, in the past the source has been compatible with these older glibc's, but a dependency addition appears to have caused an issue here. In trying to not restrict the build to taking place on older systems/compilers, currently WIP to use that dependency (c-ares) in a way that's compatible with older glibc's.

[pglombardo]

Just to add some environmental info, we're adding new grpc support and we're seeing this error in our Travis tests: https://travis-ci.org/instana/ruby-sensor/jobs/241093436

[apolcyn]

PR's that fix this should be in the next release (#11404 and 11403), thanks for filing

[jacobvosmaer]

That's great! Looking forward to 1.4

[apolcyn]

The 1.4 release was just pushed, going to close now as this should be fixed in it.

[pglombardo]

Posting here for continuity. 1.4 solved this issue for us. Unfortunately we now have an abort on process exit. This only happens in Travis test runs with the grpc gem installed.

Will research a bit or as a last resort, file a separate issue.

[apolcyn]

@pglombardo in those tests in which your seeing an abort signal, is there a server that gets shut down abruptly, e.g. the process gets a SIGINT? If so, I can reproduce this, but it should be benign (shutting down the server before process exit I believe should fix this). If this is a causing a problem feel free to file a separate issue.