search

akihikodaki / cld3-ruby

cld3-ruby is an interface of Compact Language Detector v3 (CLD3) for Ruby.
Apache License 2.0
78 stars 22 forks source link

cld3 load error with garbage as error message #9

Closed BastienDurel closed 7 years ago

BastienDurel commented 7 years ago

Hello.

I got an error while updating mastodon, and I don't know ruby well enough to debut it, but it seems it's cld3-related :

mastodon:~/live$ RAILS_ENV=production bundle exec rails assets:precompile --trace
rails aborted!
Bundler::GemRequireError: There was an error while trying to load the gem 'cld3'.
Gem Load Error is: ����t
Backtrace for gem load error is:
/usr/lib/ruby/gems/2.4.0/gems/ffi-1.9.18/lib/ffi/library.rb:277:in `attach'
/usr/lib/ruby/gems/2.4.0/gems/ffi-1.9.18/lib/ffi/library.rb:277:in `attach_function'
/usr/lib/ruby/gems/2.4.0/gems/cld3-3.1.3/lib/cld3.rb:93:in `<module:Unstable>'
/usr/lib/ruby/gems/2.4.0/gems/cld3-3.1.3/lib/cld3.rb:84:in `<module:CLD3>'
/usr/lib/ruby/gems/2.4.0/gems/cld3-3.1.3/lib/cld3.rb:24:in `<top (required)>'
/usr/lib/ruby/gems/2.4.0/gems/bootsnap-1.0.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:50:in `require'
/usr/lib/ruby/gems/2.4.0/gems/bootsnap-1.0.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:50:in `require'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/runtime.rb:82:in `block (2 levels) in require'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/runtime.rb:77:in `each'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/runtime.rb:77:in `block in require'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/runtime.rb:66:in `each'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/runtime.rb:66:in `require'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler.rb:108:in `require'
/home/mastodon/live/config/application.rb:7:in `<top (required)>'
/usr/lib/ruby/gems/2.4.0/gems/bootsnap-1.0.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:17:in `require'
/usr/lib/ruby/gems/2.4.0/gems/bootsnap-1.0.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:17:in `require'
/usr/lib/ruby/gems/2.4.0/gems/activesupport-5.1.1/lib/active_support/dependencies.rb:292:in `block in require'
/usr/lib/ruby/gems/2.4.0/gems/activesupport-5.1.1/lib/active_support/dependencies.rb:258:in `load_dependency'
/usr/lib/ruby/gems/2.4.0/gems/activesupport-5.1.1/lib/active_support/dependencies.rb:292:in `require'
/home/mastodon/live/Rakefile:4:in `<top (required)>'
/usr/lib/ruby/gems/2.4.0/gems/bootsnap-1.0.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `load'
/usr/lib/ruby/gems/2.4.0/gems/bootsnap-1.0.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `load'
/usr/lib/ruby/gems/2.4.0/gems/activesupport-5.1.1/lib/active_support/dependencies.rb:286:in `block in load'
/usr/lib/ruby/gems/2.4.0/gems/activesupport-5.1.1/lib/active_support/dependencies.rb:258:in `load_dependency'
/usr/lib/ruby/gems/2.4.0/gems/activesupport-5.1.1/lib/active_support/dependencies.rb:286:in `load'
/usr/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/rake_module.rb:28:in `load_rakefile'
/usr/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:687:in `raw_load_rakefile'
/usr/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:96:in `block in load_rakefile'
/usr/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:178:in `standard_exception_handling'
/usr/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:95:in `load_rakefile'
/usr/lib/ruby/gems/2.4.0/gems/railties-5.1.1/lib/rails/commands/rake/rake_command.rb:20:in `block in perform'
/usr/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:178:in `standard_exception_handling'
/usr/lib/ruby/gems/2.4.0/gems/railties-5.1.1/lib/rails/commands/rake/rake_command.rb:18:in `perform'
/usr/lib/ruby/gems/2.4.0/gems/railties-5.1.1/lib/rails/command.rb:46:in `invoke'
/usr/lib/ruby/gems/2.4.0/gems/railties-5.1.1/lib/rails/commands.rb:16:in `<top (required)>'
/usr/lib/ruby/gems/2.4.0/gems/bootsnap-1.0.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:17:in `require'
/usr/lib/ruby/gems/2.4.0/gems/bootsnap-1.0.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:17:in `require'
/usr/lib/ruby/gems/2.4.0/gems/activesupport-5.1.1/lib/active_support/dependencies.rb:292:in `block in require'
/usr/lib/ruby/gems/2.4.0/gems/activesupport-5.1.1/lib/active_support/dependencies.rb:258:in `load_dependency'
/usr/lib/ruby/gems/2.4.0/gems/activesupport-5.1.1/lib/active_support/dependencies.rb:292:in `require'
bin/rails:4:in `<main>'
Bundler Error Backtrace:
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/runtime.rb:85:in `rescue in block (2 levels) in require'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/runtime.rb:81:in `block (2 levels) in require'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/runtime.rb:77:in `each'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/runtime.rb:77:in `block in require'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/runtime.rb:66:in `each'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/runtime.rb:66:in `require'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler.rb:108:in `require'
/home/mastodon/live/config/application.rb:7:in `<top (required)>'
/usr/lib/ruby/gems/2.4.0/gems/bootsnap-1.0.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:17:in `require'
/usr/lib/ruby/gems/2.4.0/gems/bootsnap-1.0.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:17:in `require'
/usr/lib/ruby/gems/2.4.0/gems/activesupport-5.1.1/lib/active_support/dependencies.rb:292:in `block in require'
/usr/lib/ruby/gems/2.4.0/gems/activesupport-5.1.1/lib/active_support/dependencies.rb:258:in `load_dependency'
/usr/lib/ruby/gems/2.4.0/gems/activesupport-5.1.1/lib/active_support/dependencies.rb:292:in `require'
/home/mastodon/live/Rakefile:4:in `<top (required)>'
/usr/lib/ruby/gems/2.4.0/gems/bootsnap-1.0.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `load'
/usr/lib/ruby/gems/2.4.0/gems/bootsnap-1.0.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `load'
/usr/lib/ruby/gems/2.4.0/gems/activesupport-5.1.1/lib/active_support/dependencies.rb:286:in `block in load'
/usr/lib/ruby/gems/2.4.0/gems/activesupport-5.1.1/lib/active_support/dependencies.rb:258:in `load_dependency'
/usr/lib/ruby/gems/2.4.0/gems/activesupport-5.1.1/lib/active_support/dependencies.rb:286:in `load'
/usr/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/rake_module.rb:28:in `load_rakefile'
/usr/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:687:in `raw_load_rakefile'
/usr/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:96:in `block in load_rakefile'
/usr/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:178:in `standard_exception_handling'
/usr/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:95:in `load_rakefile'
/usr/lib/ruby/gems/2.4.0/gems/railties-5.1.1/lib/rails/commands/rake/rake_command.rb:20:in `block in perform'
/usr/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:178:in `standard_exception_handling'
/usr/lib/ruby/gems/2.4.0/gems/railties-5.1.1/lib/rails/commands/rake/rake_command.rb:18:in `perform'
/usr/lib/ruby/gems/2.4.0/gems/railties-5.1.1/lib/rails/command.rb:46:in `invoke'
/usr/lib/ruby/gems/2.4.0/gems/railties-5.1.1/lib/rails/commands.rb:16:in `<top (required)>'
/usr/lib/ruby/gems/2.4.0/gems/bootsnap-1.0.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:17:in `require'
/usr/lib/ruby/gems/2.4.0/gems/bootsnap-1.0.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:17:in `require'
/usr/lib/ruby/gems/2.4.0/gems/activesupport-5.1.1/lib/active_support/dependencies.rb:292:in `block in require'
/usr/lib/ruby/gems/2.4.0/gems/activesupport-5.1.1/lib/active_support/dependencies.rb:258:in `load_dependency'
/usr/lib/ruby/gems/2.4.0/gems/activesupport-5.1.1/lib/active_support/dependencies.rb:292:in `require'
bin/rails:4:in `<main>'
����t
/usr/lib/ruby/gems/2.4.0/gems/ffi-1.9.18/lib/ffi/library.rb:277:in `attach'
/usr/lib/ruby/gems/2.4.0/gems/ffi-1.9.18/lib/ffi/library.rb:277:in `attach_function'
/usr/lib/ruby/gems/2.4.0/gems/cld3-3.1.3/lib/cld3.rb:93:in `<module:Unstable>'
/usr/lib/ruby/gems/2.4.0/gems/cld3-3.1.3/lib/cld3.rb:84:in `<module:CLD3>'
/usr/lib/ruby/gems/2.4.0/gems/cld3-3.1.3/lib/cld3.rb:24:in `<top (required)>'
/usr/lib/ruby/gems/2.4.0/gems/bootsnap-1.0.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:50:in `require'
/usr/lib/ruby/gems/2.4.0/gems/bootsnap-1.0.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:50:in `require'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/runtime.rb:82:in `block (2 levels) in require'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/runtime.rb:77:in `each'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/runtime.rb:77:in `block in require'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/runtime.rb:66:in `each'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/runtime.rb:66:in `require'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler.rb:108:in `require'
/home/mastodon/live/config/application.rb:7:in `<top (required)>'
/usr/lib/ruby/gems/2.4.0/gems/bootsnap-1.0.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:17:in `require'
/usr/lib/ruby/gems/2.4.0/gems/bootsnap-1.0.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:17:in `require'
/usr/lib/ruby/gems/2.4.0/gems/activesupport-5.1.1/lib/active_support/dependencies.rb:292:in `block in require'
/usr/lib/ruby/gems/2.4.0/gems/activesupport-5.1.1/lib/active_support/dependencies.rb:258:in `load_dependency'
/usr/lib/ruby/gems/2.4.0/gems/activesupport-5.1.1/lib/active_support/dependencies.rb:292:in `require'
/home/mastodon/live/Rakefile:4:in `<top (required)>'
/usr/lib/ruby/gems/2.4.0/gems/bootsnap-1.0.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `load'
/usr/lib/ruby/gems/2.4.0/gems/bootsnap-1.0.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `load'
/usr/lib/ruby/gems/2.4.0/gems/activesupport-5.1.1/lib/active_support/dependencies.rb:286:in `block in load'
/usr/lib/ruby/gems/2.4.0/gems/activesupport-5.1.1/lib/active_support/dependencies.rb:258:in `load_dependency'
/usr/lib/ruby/gems/2.4.0/gems/activesupport-5.1.1/lib/active_support/dependencies.rb:286:in `load'
/usr/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/rake_module.rb:28:in `load_rakefile'
/usr/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:687:in `raw_load_rakefile'
/usr/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:96:in `block in load_rakefile'
/usr/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:178:in `standard_exception_handling'
/usr/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:95:in `load_rakefile'
/usr/lib/ruby/gems/2.4.0/gems/railties-5.1.1/lib/rails/commands/rake/rake_command.rb:20:in `block in perform'
/usr/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:178:in `standard_exception_handling'
/usr/lib/ruby/gems/2.4.0/gems/railties-5.1.1/lib/rails/commands/rake/rake_command.rb:18:in `perform'
/usr/lib/ruby/gems/2.4.0/gems/railties-5.1.1/lib/rails/command.rb:46:in `invoke'
/usr/lib/ruby/gems/2.4.0/gems/railties-5.1.1/lib/rails/commands.rb:16:in `<top (required)>'
/usr/lib/ruby/gems/2.4.0/gems/bootsnap-1.0.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:17:in `require'
/usr/lib/ruby/gems/2.4.0/gems/bootsnap-1.0.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:17:in `require'
/usr/lib/ruby/gems/2.4.0/gems/activesupport-5.1.1/lib/active_support/dependencies.rb:292:in `block in require'
/usr/lib/ruby/gems/2.4.0/gems/activesupport-5.1.1/lib/active_support/dependencies.rb:258:in `load_dependency'
/usr/lib/ruby/gems/2.4.0/gems/activesupport-5.1.1/lib/active_support/dependencies.rb:292:in `require'
bin/rails:4:in `<main>'

library seems to be compiled without error : 

mastodon:~/live$ ldd vendor/bundle/ruby/2.4.0/extensions/x86_64-linux/2.4.0/cld3-3.1.2/libcld3.so 
    ldd (0x73d9397f9000)
    libprotobuf.so.11 => /usr/lib/libprotobuf.so.11 (0x73d93913f000)
    libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x73d938ded000)
    libc.musl-x86_64.so.1 => ldd (0x73d9397f9000)
    libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x73d938bdb000)
    libz.so.1 => /lib/libz.so.1 (0x73d9389c4000)
mastodon:~/live$ ldd vendor/bundle/ruby/2.4.0/extensions/x86_64-linux/2.4.0/ffi-1.9.18/ffi_c.so 
    ldd (0x651465055000)
    libruby.so.2.4 => /usr/lib/libruby.so.2.4 (0x651464a0c000)
    libffi.so.6 => /usr/lib/libffi.so.6 (0x651464804000)
    libc.musl-x86_64.so.1 => ldd (0x651465055000)
    libgmp.so.10 => /usr/lib/libgmp.so.10 (0x6514645a0000)

Can you help me debugging that ?

akihikodaki commented 7 years ago

Sure. Could you run spec of this library?

git clone --recursive https://github.com/akihikodaki/cld3-ruby.git
cd cld3-ruby
rake spec
BastienDurel commented 7 years ago

Thanks,

Here is the end of the output :

linking shared-object libcld3.so
make: Leaving directory '/root/cld3-ruby/intermediate/ext/cld3'
cd intermediate && bundle exec rspec

An error occurred while loading ./spec/cld3_spec.rb.
Failure/Error: attach_function :delete_NNetLanguageIdentifier, [ :pointer ], :void

RuntimeError:
  ?_mL@x
# ./vendor/bundle/ruby/2.4.0/gems/ffi-1.9.18/lib/ffi/library.rb:277:in `attach'
# ./vendor/bundle/ruby/2.4.0/gems/ffi-1.9.18/lib/ffi/library.rb:277:in `attach_function'
# ./lib/cld3.rb:93:in `<module:Unstable>'
# ./lib/cld3.rb:84:in `<module:CLD3>'
# ./lib/cld3.rb:24:in `<top (required)>'
# ./spec/cld3_spec.rb:20:in `require'
# ./spec/cld3_spec.rb:20:in `<top (required)>'
No examples found.

Finished in 0.00053 seconds (files took 0.14057 seconds to load)
0 examples, 0 failures, 1 error occurred outside of examples

rake aborted!
Command failed with status (1): [cd intermediate && bundle exec rspec...]
/root/cld3-ruby/Rakefile:105:in `block in <top (required)>'
Tasks: TOP => spec
(See full trace by running task with --trace)

... and the bundler output, if needed : 

Fetching gem metadata from https://rubygems.org/........
Fetching version metadata from https://rubygems.org/.
Resolving dependencies...
Fetching ffi 1.9.18
Installing ffi 1.9.18 with native extensions
Fetching rspec-support 3.6.0
Installing rspec-support 3.6.0
Fetching diff-lcs 1.3
Installing diff-lcs 1.3
Using bundler 1.15.1
Using cld3 3.1.3 from source at `.`
Fetching rspec-core 3.6.0
Installing rspec-core 3.6.0
Fetching rspec-expectations 3.6.0
Installing rspec-expectations 3.6.0
Fetching rspec-mocks 3.6.0
Installing rspec-mocks 3.6.0
Fetching rspec 3.6.0
Installing rspec 3.6.0
Bundle complete! 2 Gemfile dependencies, 9 gems now installed.
Bundled gems are installed into ./vendor/bundle.
akihikodaki commented 7 years ago

Let's take a look at the dynamic symbol table.

objdump -T /root/cld3-ruby/intermediate/cld3/libcld3.so

It should have the following entries. The number does not matter:

000000000001f3c0 g    DF .text  000000000000003a  Base        delete_NNetLanguageIdentifier
000000000001f400 g    DF .text  000000000000005e  Base        new_NNetLanguageIdentifier
000000000001f170 g    DF .text  0000000000000241  Base        NNetLanguageIdentifier_find_language
BastienDurel commented 7 years ago

While searching for rake spec on ffi github, I saw a note about mprotect errors, and running rake spec after paxctl -mc /usr/bin/ruby works

akihikodaki commented 7 years ago

I see. I'll close this issue.

akihikodaki commented 7 years ago

It seems we pushed the button at the same time xD

BastienDurel commented 7 years ago

^^