search

akihikodaki / cld3-ruby

cld3-ruby is an interface of Compact Language Detector v3 (CLD3) for Ruby.
Apache License 2.0
78 stars 22 forks source link

Debain fails to resolve _ZTIN6google8protobuf11MessageLiteE #3

Closed akihikodaki closed 7 years ago

akihikodaki commented 7 years ago

Reported at https://github.com/tootsuite/mastodon/pull/2949#issuecomment-300336939.

have error too with debian...

LoadError: Could not open library '/home/mastodon/live/vendor/bundle/ruby/2.4.0/gems/cld3-3.1.0/lib/../ext/cld3/libcld3.so': /home/mastodon/live/vendor/bundle/ruby/2.4.0/gems/cld3-3.1.0/lib/../ext/cld3/libcld3.so: undefined symbol: _ZTIN6google8protobuf11MessageLiteE

Debian 8 doesn't have g++6 (https://packages.debian.org/fr/jessie/g++) only 4.9 so the use of CLD3 is problematic

@akihikodaki already installed

root@d0p1:~# dpkg -l | grep protobuf ii libprotobuf-dev:amd64 2.6.1-1 amd64 protocol buffers C++ library (development files) ii libprotobuf-lite9:amd64 2.6.1-1 amd64 protocol buffers C++ library (lite version) ii libprotobuf9:amd64 2.6.1-1 amd64 protocol buffers C++ library ii protobuf-compiler 2.6.1-1 amd64 compiler for protocol buffer definition files

It sounds like a problem with pkg-config.

require "mkmf"

# Check pkg-config first to inform the library is missing if so.
pkg_config("protobuf")

https://github.com/akihikodaki/cld3-ruby/blob/a82866fee388c4e076f827b6aeb7e3716eaee011/ext/cld3/extconf.rb#L20

d6rkaiz commented 7 years ago

I install to libprotobuf-dev. then fix it.

beatrix-bitrot commented 7 years ago

reposting my comments from tootsuite here all together in one

i also have problems executing code after this update (attempted to execute bundle exec rails db:migrate)

eg

LoadError: Could not open library '/REDACTED/vendor/bundle/ruby/2.4.0/gems/cld3-3.1.0/lib/../ext/cld3/libcld3.so': /REDACTED/vendor/bundle/ruby/2.4.0/gems/cld3-3.1.0/lib/../ext/cld3/libcld3.so: undefined symbol: _ZTIN6google8protobuf11MessageLiteE

ldd vendor/bundle/ruby/2.4.0/gems/cld3-3.1.0/lib/libcld3.so linux-vdso.so.1 => (0x00007ffe239ed000) libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fb0b1fb9000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fb0b1cb0000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fb0b18e6000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fb0b16cf000) /lib64/ld-linux-x86-64.so.2 (0x000055fb7d4a8000)

so yeah, no protobuf it seems?

but

dpkg -l | grep pkg-config ii pkg-config 0.29.1-0ubuntu1 amd64 manage compile and link flags for libraries

i wonder what i am missing here ):

also relevant

dpkg -l | grep protobuf ii libprotobuf-dev:amd64 2.6.1-1.3 amd64 protocol buffers C++ library (development files) ii libprotobuf-lite9v5:amd64 2.6.1-1.3 amd64 protocol buffers C++ library (lite version) ii libprotobuf9v5:amd64 2.6.1-1.3 amd64 protocol buffers C++ library ii protobuf-compiler 2.6.1-1.3 amd64 compiler for protocol buffer definition files

akihikodaki commented 7 years ago

Could you tell me the result of pkg-config --modversion protobuf?

beatrix-bitrot commented 7 years ago

@akihikodaki $ pkg-config --modversion protobuf 2.6.1

akihikodaki commented 7 years ago

Hmm, so pkg-config is fine... I have no idea what is wrong. Could you upload what you have in vendor/bundle/ruby/2.4.0/extensions/x86_64-linux/2.4.0/cld3-3.1.0 or somewhere similar?

beatrix-bitrot commented 7 years ago

hmmmmmmmmmm can't right now since it's on my production server and i haven't set up any way to transfer files to and fro. i'll work something out and get back to you

beatrix-bitrot commented 7 years ago

@akihikodaki hmmm, does this work? https://transfer.sh/3Dqyw/libcld3.so

akihikodaki commented 7 years ago

@beatrix-bitrot Thanks! Do you have any other files in the directory? They would help to understand how it buillt.

akihikodaki commented 7 years ago 
$ readelf -d libcld3.so

Dynamic section at offset 0xb2d70 contains 28 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x000000000000000f (RPATH)              Library rpath: [/home/mastodon/.rbenv/versions/2.4.1/lib]
 0x000000000000000c (INIT)               0xb728
 0x000000000000000d (FINI)               0x3baf8
 0x0000000000000019 (INIT_ARRAY)         0x2af408
 0x000000000000001b (INIT_ARRAYSZ)       40 (bytes)
 0x000000000000001a (FINI_ARRAY)         0x2af430
 0x000000000000001c (FINI_ARRAYSZ)       8 (bytes)
 0x000000006ffffef5 (GNU_HASH)           0x1f0
 0x0000000000000005 (STRTAB)             0x1138
 0x0000000000000006 (SYMTAB)             0x2f8
 0x000000000000000a (STRSZ)              8081 (bytes)
 0x000000000000000b (SYMENT)             24 (bytes)
 0x0000000000000003 (PLTGOT)             0x2b3000
 0x0000000000000002 (PLTRELSZ)           2472 (bytes)
 0x0000000000000014 (PLTREL)             RELA
 0x0000000000000017 (JMPREL)             0xad80
 0x0000000000000007 (RELA)               0x3310
 0x0000000000000008 (RELASZ)             31344 (bytes)
 0x0000000000000009 (RELAENT)            24 (bytes)
 0x000000006ffffffe (VERNEED)            0x3200
 0x000000006fffffff (VERNEEDNUM)         4
 0x000000006ffffff0 (VERSYM)             0x30ca
 0x000000006ffffff9 (RELACOUNT)          1208
 0x0000000000000000 (NULL)               0x0

Looks like it is not linked with libprotobuf at all.

beatrix-bitrot commented 7 years ago

the only other file in the directory with that one is cld3.rb

akihikodaki commented 7 years ago

Probably you looked at gems directory. It should have extensions directory. For your information, I have:

ruby/2.4.0/gems/cld3-3.1.0/ext/lib has libcld3.so and cld3.rb, as you said.

beatrix-bitrot commented 7 years ago

hmm

mkmf.log contains this:

package configuration for protobuf is not found

beatrix-bitrot commented 7 years ago

gem.build_complete was empty gem_make.out had the verbose console output from building the gem, just warnings like

cc1plus: warning: command line option ‘-Wimplicit-int’ is valid for C/ObjC but not for C++ cc1plus: warning: unrecognized command line option ‘-Wno-self-assign’

and such

akihikodaki commented 7 years ago 
mkmf.log contains this:

package configuration for protobuf is not found

That is odd. Could you remove everything and reinstall? just in case something odd occurred.

cc1plus: warning: command line option ‘-Wimplicit-int’ is valid for C/ObjC but not for C++
cc1plus: warning: unrecognized command line option ‘-Wno-self-assign’

That is a fault of mkmf, which sets their C configuration to CXXFLAGS. They are just warnings and it should work anyway.

esetomo commented 7 years ago

same problem on Ubuntu 16.04.2 LTS

Could you tell me the result of pkg-config --modversion protobuf?

$ pkg-config --modversion protobuf
The program 'pkg-config' can be found in the following packages:
 * pkg-config
 * pkgconf
Try: sudo apt install <selected package>
$ sudo apt-get install pkg-config
$ gem install cld3

It became no problem.

It seems that installation was not aborted when pkg-config could not be detected.

beatrix-bitrot commented 7 years ago

@akihikodaki you were right! the one in vendor/bundle was created the first time i ran bundle install and didn't have the necessary dependencies. after that i was testing by doing gem install cld3, which installs to a different path, so the old one stayed in vendor/bundle. after removing it and bundling again everything is fine. thank you! i will copy this update into the other repo as well.

akihikodaki commented 7 years ago

@beatrix-bitrot glad to see the problem got solved. :)

tvaz commented 6 years ago

esetomo's solution just worked for me, you need to sudo apt-get install pkg-config apparently. I thought my protobuf install was broken or something. orz thanks!!