Ruby 2.0.0p0: undefined symbol: Init_jemalloc

[sriedel]

After upgrading from je 0.1.3 to jemalloc version 0.1.9, running bundle install (no problems reported during installation), when I run bundle exec je rspec spec, I get the following message:

/home/sr/.rvm/gems/ruby-2.0.0-p0/gems/bundler-1.3.4/lib/bundler/runtime.rb:72:in `require': /home/sr/.rvm/gems/ruby-2.0.0-p0/gems/jemalloc-0.1.9/lib/jemalloc.so: undefined symbol: Init_jemalloc - /home/sr/.rvm/gems/ruby-2.0.0-p0/gems/jemalloc-0.1.9/lib/jemalloc.so (LoadError)

Dumping the symbols with nm /home/sr/.rvm/gems/ruby-2.0.0-p0/gems/jemalloc-0.1.9/lib/jemalloc.so shows that indeed, there is no Init_jemalloc symbol in this .so file.

Running on Linux 3.8.0.

[nathany]

Same issue here on Ruby 2.1.1 with jemalloc (1.0.0) on OS X Mavericks:

/Users/nathany/.rvm/gems/ruby-2.1.1@jobber/gems/bundler-1.6.1/lib/bundler/runtime.rb:76:in 
`require': dlsym(0x1108f02a0, Init_jemalloc): symbol not found - 
/Users/nathany/.rvm/gems/ruby-2.1.1@jobber/extensions/x86_64-darwin-13/2.1.0/jemalloc-1.0.0/jemalloc.bundle 
(LoadError)

Tried nm jemalloc.bundle to also get a list of symbols, which doesn't include Init_jemalloc.

➜  jemalloc-1.0.0  nm jemalloc.bundle | grep jemalloc
0000000000025440 t _jemalloc_constructor
0000000000026480 t _jemalloc_postfork_child
0000000000026410 t _jemalloc_postfork_parent
00000000000263a0 t _jemalloc_prefork

[kbaum]

Same issue deploying to heroku.

[kzk]

Unfortunately I couldn't reproduce on my PC hmm... (ruby 2.1.2p95, Mac OS v10.9.3)

[nathany]

OS X here too. Do I need to brew install something?

[canweriotnow]

@nathany I tried brew install jemalloc, no joy.

Homebrew installed it, I just mean still can't get it to run.

[subelsky]

FWIW I have this working on Heroku in Ruby 2.1 in a non-Rails Sidekiq project. (totally saved us a ton of memory so thank you @kzk!)

[kiyoto]

@subelsky Interesting. (I am nominally involved in this project as one of the maintainers of Fluentd, which uses this gem when packaged as td-agent). So, basically you dropped jemalloc gem in and your Sidekiq app on Heroku began using less dynos? Do you have ballpark numbers?

[nathany]

@subelsky So perhaps this works on Linux but not Mac? I might have to try it in staging.

[heaven]

Same here, linux x86_64 and ruby 2.1.1/2.1.2

UPD: just tried system jemalloc package but get the same problem:

[heaven@arch: crm$] LD_PRELOAD=/usr/lib/libjemalloc.so rails server
/home/heaven/.rvm/gems/ruby-2.1.2@global/gems/bundler-1.7.3/lib/bundler/runtime.rb:76:in `require': /home/heaven/.rvm/gems/ruby-2.1.2@carmen/extensions/x86_64-linux/2.1.0/jemalloc-1.0.0/jemalloc.so: undefined symbol: Init_jemalloc - /home/heaven/.rvm/gems/ruby-2.1.2@carmen/extensions/x86_64-linux/2.1.0/jemalloc-1.0.0/jemalloc.so (LoadError)
...

UPD2: didn't notice it still uses the lib from the gem. Just uninstalled the gem and system libjemalloc.so works perfectly.

[subelsky]

@kiyoto We have an app that allocates a whole lot of short strings, that was using and releasing up to 3-4 GB of memory in some cases. This gem cut that roughly in half. I then used https://github.com/SamSaffron/memory_profiler to find the offending strings and turned 'em into immutable constants which cut us all the way down to 350 MB or so.

[canweriotnow]

@subelsky can you get it to execute locally? I'm wondering if maybe this is an OS X issue...

[subelsky]

@canweriotnow yeah I can start it locally on OS X 10.9.4

[akitaonrails]

Had the very same issue with Ruby 2.2.0 on a Ubuntu Precise 64

$ bundle exec je ./script/rails s                                                                                        [ruby-2.2.0]
/home/vagrant/.rvm/gems/ruby-2.2.0/gems/activesupport-3.2.19/lib/active_support/values/time_zone.rb:270: warning: circular argument reference - now
/home/vagrant/.rvm/gems/ruby-2.2.0@global/gems/bundler-1.7.9/lib/bundler/runtime.rb:76:in `require': /home/vagrant/.rvm/gems/ruby-2.2.0/extensions/x86_64-linux/2.2.0/jemalloc-1.0.0/jemalloc.so: undefined symbol: Init_jemalloc - /home/vagrant/.rvm/gems/ruby-2.2.0/extensions/x86_64-linux/2.2.0/jemalloc-1.0.0/jemalloc.so (LoadError)
    from /home/vagrant/.rvm/gems/ruby-2.2.0@global/gems/bundler-1.7.9/lib/bundler/runtime.rb:76:in `block (2 levels) in require'
    from /home/vagrant/.rvm/gems/ruby-2.2.0@global/gems/bundler-1.7.9/lib/bundler/runtime.rb:72:in `each'
    from /home/vagrant/.rvm/gems/ruby-2.2.0@global/gems/bundler-1.7.9/lib/bundler/runtime.rb:72:in `block in require'
    from /home/vagrant/.rvm/gems/ruby-2.2.0@global/gems/bundler-1.7.9/lib/bundler/runtime.rb:61:in `each'
    from /home/vagrant/.rvm/gems/ruby-2.2.0@global/gems/bundler-1.7.9/lib/bundler/runtime.rb:61:in `require'
    from /home/vagrant/.rvm/gems/ruby-2.2.0@global/gems/bundler-1.7.9/lib/bundler.rb:134:in `require'
    from /vagrant/codeminer42/AkitaOnRails.com/config/application.rb:13:in `<top (required)>'
    from /home/vagrant/.rvm/gems/ruby-2.2.0/gems/railties-3.2.19/lib/rails/commands.rb:53:in `require'
    from /home/vagrant/.rvm/gems/ruby-2.2.0/gems/railties-3.2.19/lib/rails/commands.rb:53:in `block in <top (required)>'
    from /home/vagrant/.rvm/gems/ruby-2.2.0/gems/railties-3.2.19/lib/rails/commands.rb:50:in `tap'
    from /home/vagrant/.rvm/gems/ruby-2.2.0/gems/railties-3.2.19/lib/rails/commands.rb:50:in `<top (required)>'
    from ./script/rails:6:in `require'
    from ./script/rails:6:in `<main>'

[nathany]

You might not need this Gem in Ruby 2.2.0 anymore:

Another feature related to memory management is an additional option for configure.in to use jemalloc Feature #9113. This feature is still experimental and currently disabled by default until we gather performance data and more use cases. When we are convinced of the benefits, this feature will be enabled by default. https://www.ruby-lang.org/en/news/2014/12/25/ruby-2-2-0-released/