search

choria-legacy / mcollective-choria

Distribution of plugins for MCollective as found in Puppet 6
Apache License 2.0
56 stars 24 forks source link

Install EM in AIO on Ubuntu 16.04 fails #58

Closed Tuxdiver closed 7 years ago

Tuxdiver commented 8 years ago

I tried to set up mcollective with your module, but the compilation of the eventmachine during the installation of the nats gem seems to produce some wrong code or there is a runtime lib missing.

Starting the mcollectived after the module run: `# /opt/puppetlabs/puppet/bin/mcollectived --config=/etc/puppetlabs/mcollective/server.cfg --pidfile=/var/run/puppetlabs/mcollective.pid --no-daemonize

/opt/puppetlabs/puppet/bin/ruby: symbol lookup error: /opt/puppetlabs/puppet/lib/ruby/gems/2.1.0/extensions/x86_64-linux/2.1.0/eventmachine-1.2.0.1/rubyeventmachine.so: undefined symbol: _ZNSt7__cxx1112basic_stringIcSt11chartraitsIcESaIcEEC1EPKcRKS3`

From my old days of c++ development in the last millenium, it looks like there is a C++ lib missing...

I tried to remove the gem, install it by hand using/opt/puppetlabs/puppet/bin/gem install nats to use the right ruby version, but that does not help.

Any clues?

Some Facts: `# facter os ruby

{ architecture => "amd64", distro => { codename => "xenial", description => "Ubuntu 16.04.1 LTS", id => "Ubuntu", release => { full => "16.04", major => "16.04" } }, family => "Debian", hardware => "x86_64", name => "Ubuntu", release => { full => "16.04", major => "16.04" }, selinux => { enabled => false } } ruby => { platform => "x86_64-linux", sitedir => "/opt/puppetlabs/puppet/lib/ruby/site_ruby/2.1.0", version => "2.1.9" } `

ripienaar commented 8 years ago

that's weird, cant imagine thats even a valid symbol anywhere.

I did test this on Ubuntu but dont think it was LTS, will have to do some tests, but basically I have no idea :)

ripienaar commented 8 years ago

@Tuxdiver can you try ldd-r rubyeventmachine.so?

dnlsng commented 8 years ago

I have noticed this too on Ubuntu 16.04. It was working fine for me on 14.04 but on 16.04 I get the same error.

It looks like the AIO puppet-agent on 16.04 includes it's own libstdc++.so.6 which wasn't present on 14.04 I think.

$ ldd /opt/puppetlabs/puppet/lib/ruby/gems/2.1.0/extensions/x86_64-linux/2.1.0/eventmachine-1.2.0.1/rubyeventmachine.so
    linux-vdso.so.1 =>  (0x00007fffeb3c5000)
    libruby.so.2.1 => /opt/puppetlabs/puppet/lib/libruby.so.2.1 (0x00007f9395132000)
    libssl.so.1.0.0 => /opt/puppetlabs/puppet/lib/libssl.so.1.0.0 (0x00007f9394ec6000)
    libcrypto.so.1.0.0 => /opt/puppetlabs/puppet/lib/libcrypto.so.1.0.0 (0x00007f9394ae8000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f93948a7000)
    libstdc++.so.6 => /opt/puppetlabs/puppet/lib/libstdc++.so.6 (0x00007f9394577000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f93941ad000)
    libgcc_s.so.1 => /opt/puppetlabs/puppet/lib/libgcc_s.so.1 (0x00007f9393f97000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f9393d93000)
    libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f9393b5a000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f9393851000)
    /lib64/ld-linux-x86-64.so.2 (0x000055ad1076b000)
$ ls -l /opt/puppetlabs/puppet/lib/libstdc++.so.6
lrwxrwxrwx 1 root root 19 Sep  1 20:40 /opt/puppetlabs/puppet/lib/libstdc++.so.6 -> libstdc++.so.6.0.18

Here's the output you requested from @Tuxdiver on my host:

$ ldd -r /opt/puppetlabs/puppet/lib/ruby/gems/2.1.0/extensions/x86_64-linux/2.1.0/eventmachine-1.2.0.1/rubyeventmachine.so
    linux-vdso.so.1 =>  (0x00007ffef83ef000)
    libruby.so.2.1 => /opt/puppetlabs/puppet/lib/libruby.so.2.1 (0x00007f320f5ad000)
    libssl.so.1.0.0 => /opt/puppetlabs/puppet/lib/libssl.so.1.0.0 (0x00007f320f341000)
    libcrypto.so.1.0.0 => /opt/puppetlabs/puppet/lib/libcrypto.so.1.0.0 (0x00007f320ef63000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f320ed22000)
    libstdc++.so.6 => /opt/puppetlabs/puppet/lib/libstdc++.so.6 (0x00007f320e9f2000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f320e628000)
    libgcc_s.so.1 => /opt/puppetlabs/puppet/lib/libgcc_s.so.1 (0x00007f320e412000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f320e20e000)
    libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f320dfd5000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f320dccc000)
    /lib64/ld-linux-x86-64.so.2 (0x000055826ec9f000)
undefined symbol: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEC1Ev (/opt/puppetlabs/puppet/lib/ruby/gems/2.1.0/extensions/x86_64-linux/2.1.0/eventmachine-1.2.0.1/rubyeventmachine.so)
undefined symbol: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEC1EPKcRKS3_  (/opt/puppetlabs/puppet/lib/ruby/gems/2.1.0/extensions/x86_64-linux/2.1.0/eventmachine-1.2.0.1/rubyeventmachine.so)
undefined symbol: _ZNSt13runtime_errorC1ERKS_   (/opt/puppetlabs/puppet/lib/ruby/gems/2.1.0/extensions/x86_64-linux/2.1.0/eventmachine-1.2.0.1/rubyeventmachine.so)
undefined symbol: _ZNSt13runtime_errorC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE (/opt/puppetlabs/puppet/lib/ruby/gems/2.1.0/extensions/x86_64-linux/2.1.0/eventmachine-1.2.0.1/rubyeventmachine.so)
undefined symbol: _ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE6lengthEv   (/opt/puppetlabs/puppet/lib/ruby/gems/2.1.0/extensions/x86_64-linux/2.1.0/eventmachine-1.2.0.1/rubyeventmachine.so)
undefined symbol: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEaSEPKc   (/opt/puppetlabs/puppet/lib/ruby/gems/2.1.0/extensions/x86_64-linux/2.1.0/eventmachine-1.2.0.1/rubyeventmachine.so)
undefined symbol: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEED1Ev (/opt/puppetlabs/puppet/lib/ruby/gems/2.1.0/extensions/x86_64-linux/2.1.0/eventmachine-1.2.0.1/rubyeventmachine.so)
undefined symbol: _ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE5c_strEv    (/opt/puppetlabs/puppet/lib/ruby/gems/2.1.0/extensions/x86_64-linux/2.1.0/eventmachine-1.2.0.1/rubyeventmachine.so)
ripienaar commented 8 years ago

add -r please?

dnlsng commented 8 years ago

I've just edited my previous comment with the output of the -r flag.

Tuxdiver commented 8 years ago

Here is mine with "-r":

# ldd -r /opt/puppetlabs/puppet/lib/ruby/gems/2.1.0/extensions/x86_64-linux/2.1.0/eventmachine-1.2.0.1/rubyeventmachine.so

    linux-vdso.so.1 =>  (0x00007fffa1fb1000)
    libruby.so.2.1 => /opt/puppetlabs/puppet/lib/libruby.so.2.1 (0x00007fa0699d1000)
    libcrypto.so.1.0.0 => /opt/puppetlabs/puppet/lib/libcrypto.so.1.0.0 (0x00007fa0695f4000)
    libssl.so.1.0.0 => /opt/puppetlabs/puppet/lib/libssl.so.1.0.0 (0x00007fa069387000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fa06915f000)
    libstdc++.so.6 => /opt/puppetlabs/puppet/lib/libstdc++.so.6 (0x00007fa068e2f000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa068a65000)
    libgcc_s.so.1 => /opt/puppetlabs/puppet/lib/libgcc_s.so.1 (0x00007fa06884f000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fa06864b000)
    libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007fa068412000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fa068109000)
    /lib64/ld-linux-x86-64.so.2 (0x000056036fe0b000)
undefined symbol: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEC1Ev (/opt/puppetlabs/puppet/lib/ruby/gems/2.1.0/extensions/x86_64-linux/2.1.0/eventmachine-1.2.0.1/rubyeventmachine.so)
undefined symbol: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEC1EPKcRKS3_  (/opt/puppetlabs/puppet/lib/ruby/gems/2.1.0/extensions/x86_64-linux/2.1.0/eventmachine-1.2.0.1/rubyeventmachine.so)
undefined symbol: _ZNSt13runtime_errorC1ERKS_   (/opt/puppetlabs/puppet/lib/ruby/gems/2.1.0/extensions/x86_64-linux/2.1.0/eventmachine-1.2.0.1/rubyeventmachine.so)
undefined symbol: _ZNSt13runtime_errorC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE (/opt/puppetlabs/puppet/lib/ruby/gems/2.1.0/extensions/x86_64-linux/2.1.0/eventmachine-1.2.0.1/rubyeventmachine.so)
undefined symbol: _ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE6lengthEv   (/opt/puppetlabs/puppet/lib/ruby/gems/2.1.0/extensions/x86_64-linux/2.1.0/eventmachine-1.2.0.1/rubyeventmachine.so)
undefined symbol: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEaSEPKc   (/opt/puppetlabs/puppet/lib/ruby/gems/2.1.0/extensions/x86_64-linux/2.1.0/eventmachine-1.2.0.1/rubyeventmachine.so)
undefined symbol: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEED1Ev (/opt/puppetlabs/puppet/lib/ruby/gems/2.1.0/extensions/x86_64-linux/2.1.0/eventmachine-1.2.0.1/rubyeventmachine.so)
undefined symbol: _ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE5c_strEv    (/opt/puppetlabs/puppet/lib/ruby/gems/2.1.0/extensions/x86_64-linux/2.1.0/eventmachine-1.2.0.1/rubyeventmachine.so)

the same.

If I use LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libstdc++.so.6 before the command:

# LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libstdc++.so.6 ldd -r /opt/puppetlabs/puppet/lib/ruby/gems/2.1.0/extensions/x86_64-linux/2.1.0/eventmachine-1.2.0.1/rubyeventmachine.so
    linux-vdso.so.1 =>  (0x00007fff09dd2000)
    /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f6a7c596000)
    libruby.so.2.1 => /opt/puppetlabs/puppet/lib/libruby.so.2.1 (0x00007f6a7c0c5000)
    libcrypto.so.1.0.0 => /opt/puppetlabs/puppet/lib/libcrypto.so.1.0.0 (0x00007f6a7bce7000)
    libssl.so.1.0.0 => /opt/puppetlabs/puppet/lib/libssl.so.1.0.0 (0x00007f6a7ba7b000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f6a7b853000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f6a7b489000)
    libgcc_s.so.1 => /opt/puppetlabs/puppet/lib/libgcc_s.so.1 (0x00007f6a7b273000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f6a7af6a000)
    /lib64/ld-linux-x86-64.so.2 (0x000055b9a8217000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f6a7ad65000)
    libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f6a7ab2d000)

it works - but that's no solution :-)

ripienaar commented 8 years ago

so I wonder if during compilation of eventmachine we must point it at the right place for these libs so there is no mix of versions involved?

Tuxdiver commented 8 years ago

That sounds good - but no ruby expert I am... Don't know how to tell "gem" / "g++" to not use the default paths from /usr/lib/...

ripienaar commented 8 years ago

you can pass --with-cppflags and --with-ldflags to gem install to tweak those paths during build

Tuxdiver commented 8 years ago

looking at the mkmf.log shows a lot of parameters - and also the ones I would have added:

"gcc -o conftest -I/opt/puppetlabs/puppet/include/ruby-2.1.0/x86_64-linux -I/opt/puppetlabs/puppet/include/ruby-2.1.0/ruby/backward -I/opt/puppetlabs/puppet/include/ruby-2.1.0 -I. -I/opt/puppetlabs/puppet/include -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wno-unused-parameter -Wno-parentheses -Wno-long-long -Wno-missing-field-initializers -Wunused-variable -Wpointer-arith -Wwrite-strings -Wdeclaration-after-statement -Wimplicit-function-declaration -fPIC conftest.c -L. -L/opt/puppetlabs/puppet/lib -Wl,-R/opt/puppetlabs/puppet/lib -L/opt/puppetlabs/puppet/lib -Wl,-R/opt/puppetlabs/puppet/lib -L. -fstack-protector -rdynamic -Wl,-export-dynamic -L/opt/puppetlabs/puppet/lib -Wl,-R/opt/puppetlabs/puppet/lib -Wl,-R -Wl,/opt/puppetlabs/puppet/lib -L/opt/puppetlabs/puppet/lib -lruby -lpthread -ldl -lcrypt -lm -lc"

I would have added the "-L/opt/puppetlabs/puppet/lib" , but thats already there.

Any tipps, what to do? An additional "-static" to create a static linked file did not work.

ripienaar commented 8 years ago

hmm, low on ideas, will need some time to play around and try a few things, maybe tonight or tomorrow

Tuxdiver commented 8 years ago

Bad Hack: after building the gem, go to the gem dir and build the rubyeventmachine.so by hand:

root@v-puppet-1604 (Puppet 16.04):/opt/puppetlabs/puppet/lib/ruby/gems/2.1.0/gems/eventmachine-1.2.0.1/ext# g++ -shared -o rubyeventmachine.so binder.o em.o pipe.o page.o kb.o rubymain.o ed.o ssl.o cmain.o -L/usr/lib/ -L/usr/lib/x86_64-linux-gnu -Wl,-R/usr/lib -Wl,-R/usr/lib/x86_64-linux-gnu   -L. -L/opt/puppetlabs/puppet/lib -Wl,-R/opt/puppetlabs/puppet/lib -L/opt/puppetlabs/puppet/lib -Wl,-R/opt/puppetlabs/puppet/lib -L/usr/lib/ -L/opt/puppetlabs/puppet/lib  -Wl,-R/opt/puppetlabs/puppet/lib     -Wl,-R -Wl,/opt/puppetlabs/puppet/lib -L/opt/puppetlabs/puppet/lib -lruby  -lcrypto -lssl -lpthread -ldl -lcrypt -lm   -lc

root@v-puppet-1604 (Puppet 16.04):/opt/puppetlabs/puppet/lib/ruby/gems/2.1.0/gems/eventmachine-1.2.0.1/ext# ldd -r rubyeventmachine.so
./rubyeventmachine.so: /opt/puppetlabs/puppet/lib/libcrypto.so.1.0.0: no version information available (required by ./rubyeventmachine.so)
./rubyeventmachine.so: /opt/puppetlabs/puppet/lib/libssl.so.1.0.0: no version information available (required by ./rubyeventmachine.so)
    linux-vdso.so.1 =>  (0x00007ffc43d49000)
    libruby.so.2.1 => /opt/puppetlabs/puppet/lib/libruby.so.2.1 (0x00007fceb26a2000)
    libcrypto.so.1.0.0 => /opt/puppetlabs/puppet/lib/libcrypto.so.1.0.0 (0x00007fceb22c5000)
    libssl.so.1.0.0 => /opt/puppetlabs/puppet/lib/libssl.so.1.0.0 (0x00007fceb2058000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fceb1e30000)
    libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fceb1aae000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fceb16e4000)
    libgcc_s.so.1 => /opt/puppetlabs/puppet/lib/libgcc_s.so.1 (0x00007fceb14ce000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fceb12ca000)
    libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007fceb1091000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fceb0d88000)
    /lib64/ld-linux-x86-64.so.2 (0x00005596ddde7000)

I will try to get theses Parameter -L/usr/lib/ -L/usr/lib/x86_64-linux-gnu -Wl,-R/usr/lib -Wl,-R/usr/lib/x86_64-linux-gnuto the gem install call

ripienaar commented 8 years ago

Yikes, PL did indicate they might ship EM for us as part of AIO which help a real huge amount, bit slow getting them to do anything atm though

Tuxdiver commented 8 years ago

OK, I will wait for that - did not succeed in building the gem with the right paths. Maybe someone else could find a way... I will continue with rabbitmq as middleware at the moment...

ripienaar commented 8 years ago

@Tuxdiver you should for sure be able to use this module with rabbitmq, need to supply the config via hiera a bit more verbosely but you should be able to get the benefits of easier security etc this module bring just not NATS

let me know if you need help - and if you get it going a doc contribution here would be awesome

ripienaar commented 8 years ago

I can't figure out a way to fix this gem install, opened https://tickets.puppetlabs.com/browse/PA-577 upstream to see if they have ideas

ripienaar commented 7 years ago

Please follow #48 for updates on this, we're more likely to get a non EM NATS client than Puppet fixing the root problem, so I will close this

ripienaar commented 7 years ago

As of version 0.0.10 that I just released this will now not need Event Machine anymore and will work on Ubuntu /cc @Tuxdiver, @dnlsng