search

chuckremes / ffi-rzmq

FFI bindings for ZeroMQ so the library can be used under JRuby and other FFI-compliant ruby runtimes
242 stars 62 forks source link

segmentation fault in REP server #118

Closed notEthan closed 9 years ago

notEthan commented 9 years ago

I'm getting the below segfault when I run a server. the source of the server can be found here: https://github.com/notEthan/zy/blob/5fabfce565ecbe0b3cbb99f59d8c08f69b6b10a1/lib/zy/server.rb#L59-L66 the most immediately relevant lines are highlighted.

a message with two frames come in - the first one appears to be copied ('zy 0.0 json') but it segfaults somewhere between copying that and receiving the next frame.

It runs fine on my local machine (macos) but fails on an ubuntu server. 

D, [2015-02-06T21:24:35.406898 #30126] DEBUG -- : {"server_socket":"creating"}
D, [2015-02-06T21:24:35.407636 #30126] DEBUG -- : {"server_socket":"created (ZMQ::REP)"}
D, [2015-02-06T21:24:35.407735 #30126] DEBUG -- : {"server_socket":{"curve":"setting to server"}}
D, [2015-02-06T21:24:35.407857 #30126] DEBUG -- : {"server_socket":{"curve":"set to server"}}
D, [2015-02-06T21:24:35.407962 #30126] DEBUG -- : {"server_socket":{"curve":"setting private key"}}
D, [2015-02-06T21:24:35.408083 #30126] DEBUG -- : {"server_socket":{"curve":"set private key"}}
D, [2015-02-06T21:24:35.408174 #30126] DEBUG -- : {"server_socket":"binding tcp://*:5113"}
D, [2015-02-06T21:24:35.408571 #30126] DEBUG -- : {"server_socket":"bound tcp://*:5113"}
D, [2015-02-06T21:24:35.408704 #30126] DEBUG -- : {"server_socket":"ready to recv"}
D, [2015-02-06T21:26:24.637164 #30126] DEBUG -- : {"server_socket":"recvd (11)"}
D, [2015-02-06T21:26:24.637341 #30126] DEBUG -- : {"server_socket":"copied zy 0.0 json"}
/usr/local/rvm/gems/ruby-2.0.0-p353@venue/bundler/gems/zy-5fabfce565ec/lib/zy/server.rb:61: [BUG] Segmentation fault
ruby 2.0.0p353 (2013-11-22 revision 43784) [x86_64-linux]

-- Control frame information -----------------------------------------------
c:0005 p:0107 s:0027 e:000023 BLOCK  /usr/local/rvm/gems/ruby-2.0.0-p353@venue/bundler/gems/zy-5fabfce565ec/lib/zy/server.rb:61 [FINISH]
c:0004 p:---- s:0014 e:000013 CFUNC  :loop
c:0003 p:0537 s:0011 e:000010 METHOD /usr/local/rvm/gems/ruby-2.0.0-p353@venue/bundler/gems/zy-5fabfce565ec/lib/zy/server.rb:54
c:0002 p:0164 s:0004 E:001cd8 EVAL   bin/venue.rb:16 [FINISH]
c:0001 p:0000 s:0002 E:001e48 TOP    [FINISH]

bin/venue.rb:16:in `<main>'
/usr/local/rvm/gems/ruby-2.0.0-p353@venue/bundler/gems/zy-5fabfce565ec/lib/zy/server.rb:54:in `start'
/usr/local/rvm/gems/ruby-2.0.0-p353@venue/bundler/gems/zy-5fabfce565ec/lib/zy/server.rb:54:in `loop'
/usr/local/rvm/gems/ruby-2.0.0-p353@venue/bundler/gems/zy-5fabfce565ec/lib/zy/server.rb:61:in `block in start'

-- C level backtrace information -------------------------------------------

please let me know if there is additional information I can give or if I appear to be doing something wrong.

notEthan commented 9 years ago

I am also getting a segfault in the REQ client to this server. this is running on the same machine as the server. this is from an irb session. the relevant source is here: https://github.com/notEthan/yz/blob/7b9ec0b/lib/yz/client.rb#L29-L36

> client = Yz::Client.new(:connect => 'tcp://localhost:5113', server_public_key: "[redacted]", client_public_key: "[redacted]", client_private_key: "[redacted]")
*** glibc detected *** irb: double free or corruption (out): 0x0000000001a921e0 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x7eb96)[0x7fc97127ab96]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(onig_region_free+0x2c)[0x7fc9716d5ecc]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x839e6)[0x7fc97163f9e6]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x8414f)[0x7fc97164014f]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x854c8)[0x7fc9716414c8]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x30151)[0x7fc9715ec151]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(rb_ary_push+0x38)[0x7fc9715f17a8]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x86a6c)[0x7fc971642a6c]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x878c7)[0x7fc9716438c7]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(st_foreach_check+0x95)[0x7fc9716f1595]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x87e0f)[0x7fc971643e0f]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(rb_ensure+0xdb)[0x7fc97162975b]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(rb_hash_foreach+0x36)[0x7fc971644aa6]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x88c0b)[0x7fc971644c0b]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x1898d6)[0x7fc9717458d6]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x18de12)[0x7fc971749e12]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x193a16)[0x7fc97174fa16]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x195d3b)[0x7fc971751d3b]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x195f05)[0x7fc971751f05]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x195f6d)[0x7fc971751f6d]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x73e82)[0x7fc97162fe82]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x1898d6)[0x7fc9717458d6]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x18de12)[0x7fc971749e12]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x193a16)[0x7fc97174fa16]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x199516)[0x7fc971755516]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(rb_rescue2+0x17b)[0x7fc9716292eb]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x1877ee)[0x7fc9717437ee]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x1898d6)[0x7fc9717458d6]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x196ee7)[0x7fc971752ee7]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x18dd5b)[0x7fc971749d5b]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x193a16)[0x7fc97174fa16]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x195d3b)[0x7fc971751d3b]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x199cdb)[0x7fc971755cdb]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(rb_catch_obj+0xc6)[0x7fc971746e66]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x18af1e)[0x7fc971746f1e]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x1898d6)[0x7fc9717458d6]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x196ee7)[0x7fc971752ee7]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x18dd5b)[0x7fc971749d5b]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x193a16)[0x7fc97174fa16]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x195d3b)[0x7fc971751d3b]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x199cdb)[0x7fc971755cdb]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(rb_catch_obj+0xc6)[0x7fc971746e66]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x18af1e)[0x7fc971746f1e]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x1898d6)[0x7fc9717458d6]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x196ee7)[0x7fc971752ee7]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x18dd5b)[0x7fc971749d5b]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x193a16)[0x7fc97174fa16]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(rb_iseq_eval_main+0x8a)[0x7fc97175663a]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(+0x6bac2)[0x7fc971627ac2]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(ruby_exec_node+0x1d)[0x7fc971628dad]
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/../lib/libruby.so.2.0(ruby_run_node+0x1e)[0x7fc97162afae]
irb[0x40080b]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed)[0x7fc97121d76d]
irb[0x400839]
======= Memory map: ========
00400000-00401000 r-xp 00000000 ca:01 396121                             /usr/local/rvm/rubies/ruby-2.0.0-p353/bin/ruby
00600000-00601000 r--p 00000000 ca:01 396121                             /usr/local/rvm/rubies/ruby-2.0.0-p353/bin/ruby
00601000-00602000 rw-p 00001000 ca:01 396121                             /usr/local/rvm/rubies/ruby-2.0.0-p353/bin/ruby
011ff000-01c01000 rw-p 00000000 00:00 0                                  [heap]
7fc960000000-7fc960021000 rw-p 00000000 00:00 0 
7fc960021000-7fc964000000 ---p 00000000 00:00 0 
7fc968000000-7fc968021000 rw-p 00000000 00:00 0 
7fc968021000-7fc96c000000 ---p 00000000 00:00 0 
7fc96cad2000-7fc96caea000 r-xp 00000000 ca:01 134857                     /lib/x86_64-linux-gnu/libresolv-2.15.so
7fc96caea000-7fc96ccea000 ---p 00018000 ca:01 134857                     /lib/x86_64-linux-gnu/libresolv-2.15.so
7fc96ccea000-7fc96cceb000 r--p 00018000 ca:01 134857                     /lib/x86_64-linux-gnu/libresolv-2.15.so
7fc96cceb000-7fc96ccec000 rw-p 00019000 ca:01 134857                     /lib/x86_64-linux-gnu/libresolv-2.15.so
7fc96ccec000-7fc96ccee000 rw-p 00000000 00:00 0 
7fc96ccee000-7fc96ccf5000 r-xp 00000000 ca:01 134839                     /lib/x86_64-linux-gnu/libnss_dns-2.15.so
7fc96ccf5000-7fc96cef4000 ---p 00007000 ca:01 134839                     /lib/x86_64-linux-gnu/libnss_dns-2.15.so
7fc96cef4000-7fc96cef5000 r--p 00006000 ca:01 134839                     /lib/x86_64-linux-gnu/libnss_dns-2.15.so
7fc96cef5000-7fc96cef6000 rw-p 00007000 ca:01 134839                     /lib/x86_64-linux-gnu/libnss_dns-2.15.so
7fc96cef6000-7fc96cf02000 r-xp 00000000 ca:01 134840                     /lib/x86_64-linux-gnu/libnss_files-2.15.so
7fc96cf02000-7fc96d101000 ---p 0000c000 ca:01 134840                     /lib/x86_64-linux-gnu/libnss_files-2.15.soAborted (core dumped)
notEthan commented 9 years ago

the client does (usually) successfully create the connection, but in this instance it segfaulted. it is not segfaulting at the same time as the server - the client segfaulted when making a connection; the server segfaulted when a connection had already been made and one message frame had been received, before the next frame had been received.

I have not yet successfully gotten a reply from the REP server to the REQ client on this machine. it works consistently on my osx machine.

notEthan commented 9 years ago

the REP server seems to segfault consistently in the same place when the client sends a message.

chuckremes commented 9 years ago

Looks like the Ruby runtime is crashing. I see that it’s MRI. What version? 2.0, 2.1, 2.2?

Also, you might want to try it with an alternate runtime such as Rubinius to see if the crashes still happen. It’s highly unlikely that it is ffi-rzmq at fault here, so more information to narrow down the problem would be useful.

On Feb 6, 2015, at 3:49 PM, notEthan notifications@github.com wrote:

the REP server seems to segfault consistently in the same place when the client sends a message.

— Reply to this email directly or view it on GitHub https://github.com/chuckremes/ffi-rzmq/issues/118#issuecomment-73318942.

notEthan commented 9 years ago

It is ruby-2.0.0-p353. I will try rubinius.

chuckremes commented 9 years ago

You might also want to try MRI 2.1 and the recently released 2.2.

On Feb 6, 2015, at 4:05 PM, notEthan notifications@github.com wrote:

It is ruby-2.0.0-p353. I will try rubinius.

— Reply to this email directly or view it on GitHub https://github.com/chuckremes/ffi-rzmq/issues/118#issuecomment-73321356.

notEthan commented 9 years ago

sad rubinius too. the client: 

rbx-2.5.2 :155 > client = Yz::Client.new(:connect => 'tcp://levis-dev-app-0.venuenext.net:5113', server_public_key: "[redacted]", client_public_key: "[redacted]", client_private_key: "[redacted]")
=> #<Yz::Client:0xb934 @options={"connect"=>"tcp://levis-dev-app-0.venuenext.net:5113", ...}> 
rbx-2.5.2 :156 > client.read(:bar, name: 'foo')
Assertion failed: check () (src/msg.cpp:226)
The Rubinius process is aborting with signal: SIGABRT
--- begin system info ---
sysname: Linux
nodename: levis-dev-app-0
release: 3.2.0-59-virtual
version: #90-Ubuntu SMP Tue Jan 7 23:02:33 UTC 2014
machine: x86_64
--- end system info ---
--- begin rubinius info ---
program name: rbx
version: 2.5.2
ruby version: 2.1.0
release date: 2015-01-30
build revision: 7a5b05b1a9aca87703ceb0bafe00b916b9510285
llvm version: 3.4
--- end rubinius info ---
--- begin system backtrace ---
irb() [0x6980da]
/lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f9a3af47cb0]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f9a3a175425]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7f9a3a178b8b]
/usr/local/lib/libzmq.so(+0x20439) [0x7f9a30a6b439]
/usr/local/lib/libzmq.so(+0x27486) [0x7f9a30a72486]
irb(ffi_call_unix64+0x4c) [0x7da6c8]
irb(ffi_call+0x1b5) [0x7da015]
/usr/local/rvm/gems/rbx-2.5.2@venue/gems/ffi-1.9.6/lib/ffi_c.so(+0xf374) [0x7f9a394cd374]
irb(rb_thread_call_without_gvl+0x6c) [0x73f80c]
/usr/local/rvm/gems/rbx-2.5.2@venue/gems/ffi-1.9.6/lib/ffi_c.so(+0xf3ac) [0x7f9a394cd3ac]
irb(rb_rescue2+0x85) [0x737605]
/usr/local/rvm/gems/rbx-2.5.2@venue/gems/ffi-1.9.6/lib/ffi_c.so(rbffi_CallFunction+0x227) [0x7f9a394cddc7]
irb() [0x6fca4c]
irb(_ZN8rubinius12NativeMethod23executor_implementationINS_16GenericArgumentsEEEPNS_6ObjectEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0x416) [0x6fddf6]
irb(_ZN8rubinius8CallSite11empty_cacheEPNS_5StateEPS0_PNS_9CallFrameERNS_9ArgumentsE+0x134) [0x6c95a4]
irb(_ZN8rubinius11MachineCode11interpreterEPNS_5StateEPS0_PNS_20InterpreterCallFrameE+0x1740) [0x5f0360]
irb(_ZN8rubinius11MachineCode19execute_specializedINS_11NoArgumentsEEEPNS_6ObjectEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0x21a) [0x63310a]
irb(_ZN8rubinius12CompiledCode16default_executorEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0xc7) [0x6ce097]
irb(_ZN8rubinius8CallSite17empty_cache_vcallEPNS_5StateEPS0_PNS_9CallFrameERNS_9ArgumentsE+0x13b) [0x6c91ab]
irb(_ZN8rubinius11MachineCode11interpreterEPNS_5StateEPS0_PNS_20InterpreterCallFrameE+0x166e) [0x5f028e]
irb(_ZN8rubinius11MachineCode19execute_specializedINS_11NoArgumentsEEEPNS_6ObjectEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0x21a) [0x63310a]
irb(_ZN8rubinius12CompiledCode16default_executorEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0xc7) [0x6ce097]
irb(_ZN8rubinius8CallSite11empty_cacheEPNS_5StateEPS0_PNS_9CallFrameERNS_9ArgumentsE+0x134) [0x6c95a4]
irb(_ZN8rubinius11MachineCode11interpreterEPNS_5StateEPS0_PNS_20InterpreterCallFrameE+0x1740) [0x5f0360]
irb(_ZN8rubinius11MachineCode19execute_specializedINS_11OneArgumentEEEPNS_6ObjectEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0x232) [0x6327f2]
irb(_ZN8rubinius12CompiledCode16default_executorEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0xc7) [0x6ce097]
irb(_ZN8rubinius8CallSite19empty_cache_privateEPNS_5StateEPS0_PNS_9CallFrameERNS_9ArgumentsE+0x134) [0x6c93a4]
irb(_ZN8rubinius11MachineCode11interpreterEPNS_5StateEPS0_PNS_20InterpreterCallFrameE+0x1740) [0x5f0360]
irb(_ZN8rubinius16BlockEnvironment19execute_interpreterEPNS_5StateEPNS_9CallFrameEPS0_RNS_9ArgumentsERNS_15BlockInvocationE+0x268) [0x6c5368]
irb(_ZN8rubinius16BlockEnvironment6invokeEPNS_5StateEPNS_9CallFrameEPS0_RNS_9ArgumentsERNS_15BlockInvocationE+0x7d) [0x6c622d]
irb(_ZN8rubinius13BlockAsMethod14block_executorEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0x15d) [0x6c4dbd]
irb(_ZN8rubinius8CallSite11empty_cacheEPNS_5StateEPS0_PNS_9CallFrameERNS_9ArgumentsE+0x134) [0x6c95a4]
irb(_ZN8rubinius11MachineCode11interpreterEPNS_5StateEPS0_PNS_20InterpreterCallFrameE+0x1740) [0x5f0360]
irb(_ZN8rubinius16BlockEnvironment19execute_interpreterEPNS_5StateEPNS_9CallFrameEPS0_RNS_9ArgumentsERNS_15BlockInvocationE+0x268) [0x6c5368]
irb(_ZN8rubinius16BlockEnvironment6invokeEPNS_5StateEPNS_9CallFrameEPS0_RNS_9ArgumentsERNS_15BlockInvocationE+0x7d) [0x6c622d]
irb(_ZN8rubinius16BlockEnvironment10call_underEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0xbb) [0x6c667b]
irb(_ZN8rubinius10Primitives16block_call_underEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0x8c) [0x67321c]
irb(_ZN8rubinius11MachineCode11interpreterEPNS_5StateEPS0_PNS_20InterpreterCallFrameE+0x3111) [0x5f1d31]
irb(_ZN8rubinius11MachineCode19execute_specializedINS_16GenericArgumentsEEEPNS_6ObjectEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0x26c) [0x630c8c]
irb(_ZN8rubinius11MachineCode11interpreterEPNS_5StateEPS0_PNS_20InterpreterCallFrameE+0x1740) [0x5f0360]
irb(_ZN8rubinius11MachineCode19execute_specializedINS_16GenericArgumentsEEEPNS_6ObjectEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0x26c) [0x630c8c]
irb(_ZN8rubinius11MachineCode11interpreterEPNS_5StateEPS0_PNS_20InterpreterCallFrameE+0x1740) [0x5f0360]
irb(_ZN8rubinius11MachineCode19execute_specializedINS_16GenericArgumentsEEEPNS_6ObjectEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0x26c) [0x630c8c]
irb(_ZN8rubinius11MachineCode11interpreterEPNS_5StateEPS0_PNS_20InterpreterCallFrameE+0x1740) [0x5f0360]
irb(_ZN8rubinius11MachineCode19execute_specializedINS_12TwoArgumentsEEEPNS_6ObjectEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0x23a) [0x63230a]
irb(_ZN8rubinius11MachineCode11interpreterEPNS_5StateEPS0_PNS_20InterpreterCallFrameE+0x1740) [0x5f0360]
irb(_ZN8rubinius16BlockEnvironment19execute_interpreterEPNS_5StateEPNS_9CallFrameEPS0_RNS_9ArgumentsERNS_15BlockInvocationE+0x268) [0x6c5368]
irb(_ZN8rubinius16BlockEnvironment6invokeEPNS_5StateEPNS_9CallFrameEPS0_RNS_9ArgumentsERNS_15BlockInvocationE+0x7d) [0x6c622d]
irb(_ZN8rubinius16BlockEnvironment4callEPNS_5StateEPNS_9CallFrameERNS_9ArgumentsEi+0x3d) [0x6c644d]
irb(_ZN8rubinius11MachineCode11interpreterEPNS_5StateEPS0_PNS_20InterpreterCallFrameE+0x43ab) [0x5f2fcb]
irb(_ZN8rubinius11MachineCode19execute_specializedINS_11OneArgumentEEEPNS_6ObjectEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0x232) [0x6327f2]
irb(_ZN8rubinius11MachineCode11interpreterEPNS_5StateEPS0_PNS_20InterpreterCallFrameE+0x181d) [0x5f043d]
irb(_ZN8rubinius16BlockEnvironment19execute_interpreterEPNS_5StateEPNS_9CallFrameEPS0_RNS_9ArgumentsERNS_15BlockInvocationE+0x268) [0x6c5368]
irb(_ZN8rubinius16BlockEnvironment6invokeEPNS_5StateEPNS_9CallFrameEPS0_RNS_9ArgumentsERNS_15BlockInvocationE+0x7d) [0x6c622d]
irb(_ZN8rubinius16BlockEnvironment4callEPNS_5StateEPNS_9CallFrameERNS_9ArgumentsEi+0x3d) [0x6c644d]
irb(_ZN8rubinius11MachineCode11interpreterEPNS_5StateEPS0_PNS_20InterpreterCallFrameE+0x43ab) [0x5f2fcb]
irb(_ZN8rubinius16BlockEnvironment19execute_interpreterEPNS_5StateEPNS_9CallFrameEPS0_RNS_9ArgumentsERNS_15BlockInvocationE+0x268) [0x6c5368]
irb(_ZN8rubinius16BlockEnvironment6invokeEPNS_5StateEPNS_9CallFrameEPS0_RNS_9ArgumentsERNS_15BlockInvocationE+0x7d) [0x6c622d]
irb(_ZN8rubinius16BlockEnvironment4callEPNS_5StateEPNS_9CallFrameERNS_9ArgumentsEi+0x3d) [0x6c644d]
irb(_ZN8rubinius11MachineCode11interpreterEPNS_5StateEPS0_PNS_20InterpreterCallFrameE+0x43ab) [0x5f2fcb]
irb(_ZN8rubinius11MachineCode19execute_specializedINS_11NoArgumentsEEEPNS_6ObjectEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0x21a) [0x63310a]
irb(_ZN8rubinius12CompiledCode16default_executorEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0xc7) [0x6ce097]
irb(_ZN8rubinius8CallSite19empty_cache_privateEPNS_5StateEPS0_PNS_9CallFrameERNS_9ArgumentsE+0x134) [0x6c93a4]
irb(_ZN8rubinius11MachineCode11interpreterEPNS_5StateEPS0_PNS_20InterpreterCallFrameE+0x181d) [0x5f043d]
irb(_ZN8rubinius16BlockEnvironment19execute_interpreterEPNS_5StateEPNS_9CallFrameEPS0_RNS_9ArgumentsERNS_15BlockInvocationE+0x268) [0x6c5368]
irb(_ZN8rubinius16BlockEnvironment6invokeEPNS_5StateEPNS_9CallFrameEPS0_RNS_9ArgumentsERNS_15BlockInvocationE+0x7d) [0x6c622d]
irb(_ZN8rubinius16BlockEnvironment4callEPNS_5StateEPNS_9CallFrameERNS_9ArgumentsEi+0x3d) [0x6c644d]
irb(_ZN8rubinius4Proc4callEPNS_5StateEPNS_9CallFrameERNS_9ArgumentsE+0x34c) [0x70ca3c]
irb(_ZN8rubinius10Primitives9proc_callEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0x8c) [0x65687c]
irb(_ZN8rubinius11MachineCode11interpreterEPNS_5StateEPS0_PNS_20InterpreterCallFrameE+0x3111) [0x5f1d31]
irb(_ZN8rubinius11MachineCode19execute_specializedINS_17SplatOnlyArgumentEEEPNS_6ObjectEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0x27d) [0x632d1d]
irb(_ZN8rubinius8Dispatch4sendEPNS_5StateEPNS_9CallFrameERNS_10LookupDataERNS_9ArgumentsENS_19MethodMissingReasonE+0x6c) [0x5c4b8c]
irb(_ZN8rubinius6System8vm_catchEPNS_5StateEPNS_6ObjectES4_PNS_9CallFrameE+0x126) [0x71bf26]
irb(_ZN8rubinius10Primitives8vm_catchEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0x6d) [0x64dabd]
irb(_ZN8rubinius11MachineCode11interpreterEPNS_5StateEPS0_PNS_20InterpreterCallFrameE+0x1740) [0x5f0360]
irb(_ZN8rubinius16BlockEnvironment19execute_interpreterEPNS_5StateEPNS_9CallFrameEPS0_RNS_9ArgumentsERNS_15BlockInvocationE+0x268) [0x6c5368]
irb(_ZN8rubinius16BlockEnvironment6invokeEPNS_5StateEPNS_9CallFrameEPS0_RNS_9ArgumentsERNS_15BlockInvocationE+0x7d) [0x6c622d]
irb(_ZN8rubinius16BlockEnvironment4callEPNS_5StateEPNS_9CallFrameERNS_9ArgumentsEi+0x3d) [0x6c644d]
irb(_ZN8rubinius11MachineCode11interpreterEPNS_5StateEPS0_PNS_20InterpreterCallFrameE+0x43ab) [0x5f2fcb]
irb(_ZN8rubinius11MachineCode19execute_specializedINS_11OneArgumentEEEPNS_6ObjectEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0x232) [0x6327f2]
irb(_ZN8rubinius11MachineCode11interpreterEPNS_5StateEPS0_PNS_20InterpreterCallFrameE+0x181d) [0x5f043d]
irb(_ZN8rubinius11MachineCode19execute_specializedINS_16GenericArgumentsEEEPNS_6ObjectEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0x26c) [0x630c8c]
irb(_ZN8rubinius8CallSite19empty_cache_privateEPNS_5StateEPS0_PNS_9CallFrameERNS_9ArgumentsE+0x134) [0x6c93a4]
irb(_ZN8rubinius11MachineCode11interpreterEPNS_5StateEPS0_PNS_20InterpreterCallFrameE+0x181d) [0x5f043d]
irb(_ZN8rubinius11MachineCode19execute_specializedINS_11NoArgumentsEEEPNS_6ObjectEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0x21a) [0x63310a]
irb(_ZN8rubinius12CompiledCode16default_executorEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0xc7) [0x6ce097]
irb(_ZN8rubinius8CallSite11empty_cacheEPNS_5StateEPS0_PNS_9CallFrameERNS_9ArgumentsE+0x134) [0x6c95a4]
irb(_ZN8rubinius11MachineCode11interpreterEPNS_5StateEPS0_PNS_20InterpreterCallFrameE+0x181d) [0x5f043d]
irb(_ZN8rubinius11MachineCode19execute_specializedINS_11NoArgumentsEEEPNS_6ObjectEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0x21a) [0x63310a]
irb(_ZN8rubinius12CompiledCode16default_executorEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0xc7) [0x6ce097]
irb(_ZN8rubinius8CallSite11empty_cacheEPNS_5StateEPS0_PNS_9CallFrameERNS_9ArgumentsE+0x134) [0x6c95a4]
irb(_ZN8rubinius11MachineCode11interpreterEPNS_5StateEPS0_PNS_20InterpreterCallFrameE+0x1740) [0x5f0360]
irb(_ZN8rubinius16BlockEnvironment19execute_interpreterEPNS_5StateEPNS_9CallFrameEPS0_RNS_9ArgumentsERNS_15BlockInvocationE+0x268) [0x6c5368]
irb(_ZN8rubinius16BlockEnvironment6invokeEPNS_5StateEPNS_9CallFrameEPS0_RNS_9ArgumentsERNS_15BlockInvocationE+0x7d) [0x6c622d]
irb(_ZN8rubinius16BlockEnvironment4callEPNS_5StateEPNS_9CallFrameERNS_9ArgumentsEi+0x3d) [0x6c644d]
irb(_ZN8rubinius4Proc4callEPNS_5StateEPNS_9CallFrameERNS_9ArgumentsE+0x34c) [0x70ca3c]
irb(_ZN8rubinius10Primitives9proc_callEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0x8c) [0x65687c]
irb(_ZN8rubinius11MachineCode11interpreterEPNS_5StateEPS0_PNS_20InterpreterCallFrameE+0x3111) [0x5f1d31]
irb(_ZN8rubinius11MachineCode19execute_specializedINS_17SplatOnlyArgumentEEEPNS_6ObjectEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0x27d) [0x632d1d]
irb(_ZN8rubinius8Dispatch4sendEPNS_5StateEPNS_9CallFrameERNS_10LookupDataERNS_9ArgumentsENS_19MethodMissingReasonE+0x6c) [0x5c4b8c]
irb(_ZN8rubinius6System8vm_catchEPNS_5StateEPNS_6ObjectES4_PNS_9CallFrameE+0x126) [0x71bf26]
irb(_ZN8rubinius10Primitives8vm_catchEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0x6d) [0x64dabd]
irb(_ZN8rubinius12CompiledCode16default_executorEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0xc7) [0x6ce097]
irb(_ZN8rubinius8CallSite11empty_cacheEPNS_5StateEPS0_PNS_9CallFrameERNS_9ArgumentsE+0x134) [0x6c95a4]
irb(_ZN8rubinius11MachineCode11interpreterEPNS_5StateEPS0_PNS_20InterpreterCallFrameE+0x1740) [0x5f0360]
irb(_ZN8rubinius16BlockEnvironment19execute_interpreterEPNS_5StateEPNS_9CallFrameEPS0_RNS_9ArgumentsERNS_15BlockInvocationE+0x268) [0x6c5368]
irb(_ZN8rubinius16BlockEnvironment6invokeEPNS_5StateEPNS_9CallFrameEPS0_RNS_9ArgumentsERNS_15BlockInvocationE+0x7d) [0x6c622d]
irb(_ZN8rubinius16BlockEnvironment4callEPNS_5StateEPNS_9CallFrameERNS_9ArgumentsEi+0x3d) [0x6c644d]
irb(_ZN8rubinius11MachineCode11interpreterEPNS_5StateEPS0_PNS_20InterpreterCallFrameE+0x43ab) [0x5f2fcb]
irb(_ZN8rubinius11MachineCode19execute_specializedINS_11OneArgumentEEEPNS_6ObjectEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0x232) [0x6327f2]
irb(_ZN8rubinius12CompiledCode16default_executorEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0xc7) [0x6ce097]
irb(_ZN8rubinius8CallSite11empty_cacheEPNS_5StateEPS0_PNS_9CallFrameERNS_9ArgumentsE+0x134) [0x6c95a4]
irb(_ZN8rubinius11MachineCode11interpreterEPNS_5StateEPS0_PNS_20InterpreterCallFrameE+0x181d) [0x5f043d]
irb(_ZN8rubinius11MachineCode19execute_specializedINS_16GenericArgumentsEEEPNS_6ObjectEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0x26c) [0x630c8c]
irb(_ZN8rubinius12CompiledCode16default_executorEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0xc7) [0x6ce097]
irb(_ZN8rubinius8CallSite19empty_cache_privateEPNS_5StateEPS0_PNS_9CallFrameERNS_9ArgumentsE+0x134) [0x6c93a4]
irb(_ZN8rubinius11MachineCode11interpreterEPNS_5StateEPS0_PNS_20InterpreterCallFrameE+0x181d) [0x5f043d]
irb(_ZN8rubinius11MachineCode19execute_specializedINS_16GenericArgumentsEEEPNS_6ObjectEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0x26c) [0x630c8c]
irb(_ZN8rubinius12CompiledCode16default_executorEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0xc7) [0x6ce097]
irb(_ZN8rubinius8CallSite11empty_cacheEPNS_5StateEPS0_PNS_9CallFrameERNS_9ArgumentsE+0x134) [0x6c95a4]
irb(_ZN8rubinius11MachineCode11interpreterEPNS_5StateEPS0_PNS_20InterpreterCallFrameE+0x1740) [0x5f0360]
irb(_ZN8rubinius11MachineCode17execute_as_scriptEPNS_5StateEPNS_12CompiledCodeEPNS_9CallFrameE+0x1ca) [0x63062a]
irb(_ZN8rubinius6System13vm_run_scriptEPNS_5StateERNS_11GCTokenImplEPNS_12CompiledCodeEPNS_9CallFrameE+0xbd) [0x71d2cd]
irb(_ZN8rubinius10Primitives13vm_run_scriptEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0xab) [0x6494bb]
irb(_ZN8rubinius11MachineCode11interpreterEPNS_5StateEPS0_PNS_20InterpreterCallFrameE+0x1740) [0x5f0360]
irb(_ZN8rubinius11MachineCode19execute_specializedINS_16GenericArgumentsEEEPNS_6ObjectEPNS_5StateEPNS_9CallFrameEPNS_10ExecutableEPNS_6ModuleERNS_9ArgumentsE+0x26c) [0x630c8c]
irb(_ZN8rubinius8CallSite19empty_cache_privateEPNS_5StateEPS0_PNS_9CallFrameERNS_9ArgumentsE+0x134) [0x6c93a4]
--- end system backtrace ---
--- begin Ruby backtraces ---
--- Thread 1 backtrace ---
0x7fff6b82cca0: capi:zmq_msg_data at Function.c
ZMQ::Message#data in /usr/local/rvm/gems/rbx-2.5.2@venue/gems/ffi-rzmq-2.0.4/lib/ffi-rzmq/message.rb:166 (+7)
ZMQ::Message#copy_out_string in /usr/local/rvm/gems/rbx-2.5.2@venue/gems/ffi-rzmq-2.0.4/lib/ffi-rzmq/message.rb:174 (+3)
Yz::Client#request in (irb):53 (+109)
__block__ in (irb):63 (+55)
__block__ in (irb):156 (+28)
Rubinius::BlockEnvironment#call_on_instance in kernel/common/block_environment.rb:53 (+90)
Kernel#eval in kernel/common/eval.rb:176 (+248)
IRB::WorkSpace#evaluate in /usr/local/rvm/gems/rbx-2.5.2@global/gems/rubysl-irb-2.1.1/lib/irb/workspace.rb:86 (+33)
IRB::Context#evaluate in /usr/local/rvm/gems/rbx-2.5.2@global/gems/rubysl-irb-2.1.1/lib/irb/context.rb:380 (+19)
__block__ in /usr/local/rvm/gems/rbx-2.5.2@global/gems/rubysl-irb-2.1.1/lib/rubysl/irb/irb.rb:492 (+25)
IRB::Irb#signal_status in /usr/local/rvm/gems/rbx-2.5.2@global/gems/rubysl-irb-2.1.1/lib/rubysl/irb/irb.rb:624 (+35)
__block__ in /usr/local/rvm/gems/rbx-2.5.2@global/gems/rubysl-irb-2.1.1/lib/rubysl/irb/irb.rb:489 (+9)
__block__ in /usr/local/rvm/gems/rbx-2.5.2@global/gems/rubysl-irb-2.1.1/lib/irb/ruby-lex.rb:247 (+106)
Kernel#loop in kernel/common/kernel.rb:510 (+27)
__block__ in /usr/local/rvm/gems/rbx-2.5.2@global/gems/rubysl-irb-2.1.1/lib/irb/ruby-lex.rb:233 (+7)
Proc#call in kernel/bootstrap/proc.rb:0 (+204)
__block__ in kernel/common/throw_catch.rb:30 (+11)
Rubinius::ThrownValue.register in kernel/common/throw_catch.rb:8 (+53)
Kernel#catch in kernel/common/throw_catch.rb:29 (+59)
RubyLex#each_top_level_statement in /usr/local/rvm/gems/rbx-2.5.2@global/gems/rubysl-irb-2.1.1/lib/irb/ruby-lex.rb:232 (+13)
IRB::Irb#eval_input in /usr/local/rvm/gems/rbx-2.5.2@global/gems/rubysl-irb-2.1.1/lib/rubysl/irb/irb.rb:488 (+28)
__block__ in /usr/local/rvm/gems/rbx-2.5.2@global/gems/rubysl-irb-2.1.1/lib/rubysl/irb/irb.rb:397 (+6)
Proc#call in kernel/bootstrap/proc.rb:0 (+204)
__block__ in kernel/common/throw_catch.rb:30 (+11)
Rubinius::ThrownValue.register in kernel/common/throw_catch.rb:8 (+53)
Kernel#catch in kernel/common/throw_catch.rb:29 (+59)
IRB.start in /usr/local/rvm/gems/rbx-2.5.2@global/gems/rubysl-irb-2.1.1/lib/rubysl/irb/irb.rb:396 (+192)
Object#__script__ in /usr/local/rvm/gems/rbx-2.5.2@global/gems/rubysl-irb-2.1.1/bin/irb:12 (+27)
Kernel#load in kernel/common/kernel.rb:497 (+58)
__block__ in /usr/local/rvm/gems/rbx-2.5.2@global/bin/irb:23 (+192)
Rubinius::BlockEnvironment#call_on_instance in kernel/common/block_environment.rb:53 (+90)
Kernel#eval in kernel/common/eval.rb:176 (+248)
Object#__script__ in /usr/local/rvm/gems/rbx-2.5.2@venue/bin/ruby_executable_hooks:15 (+177)
Rubinius::CodeLoader#load_script in kernel/delta/code_loader.rb:66 (+52)
Rubinius::CodeLoader.load_script in kernel/delta/code_loader.rb:152 (+40)
Rubinius::Loader#script in kernel/loader.rb:655 (+214)
Rubinius::Loader#main in kernel/loader.rb:809 (+77)
--- end Ruby backtraces ---
Aborted (core dumped)
notEthan commented 9 years ago

and the server:

D, [2015-02-06T22:23:24.604716 #18988] DEBUG -- : {"server_socket":"creating"}
D, [2015-02-06T22:23:24.617996 #18988] DEBUG -- : {"server_socket":"created (ZMQ::REP)"}
D, [2015-02-06T22:23:24.618347 #18988] DEBUG -- : {"server_socket":{"curve":"setting to server"}}
D, [2015-02-06T22:23:24.618715 #18988] DEBUG -- : {"server_socket":{"curve":"set to server"}}
D, [2015-02-06T22:23:24.619003 #18988] DEBUG -- : {"server_socket":{"curve":"setting private key"}}
D, [2015-02-06T22:23:24.619338 #18988] DEBUG -- : {"server_socket":{"curve":"set private key"}}
D, [2015-02-06T22:23:24.619615 #18988] DEBUG -- : {"server_socket":"binding tcp://*:5113"}
D, [2015-02-06T22:23:24.620111 #18988] DEBUG -- : {"server_socket":"bound tcp://*:5113"}
D, [2015-02-06T22:23:24.620414 #18988] DEBUG -- : {"server_socket":"ready to recv"}
D, [2015-02-06T22:25:27.523648 #18988] DEBUG -- : {"server_socket":"recvd (11)"}
D, [2015-02-06T22:25:27.524000 #18988] DEBUG -- : {"server_socket":"copied zy 0.0 json"}
D, [2015-02-06T22:25:27.524353 #18988] DEBUG -- : {"server_socket":"recvd (2)"}
D, [2015-02-06T22:25:27.524539 #18988] DEBUG -- : {"server_socket":"copied {}"}
*** glibc detected *** ruby: free(): invalid next size (fast): 0x000000000279a2c0 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x7eb96)[0x7f53e87f0b96]
/usr/local/rvm/gems/rbx-2.5.2/gems/ffi-1.9.6/lib/ffi_c.so(+0x1537e)[0x7f53c633f37e]
ruby(_ZN8rubinius16FinalizerHandler8finalizeEPNS_5StateE+0x229)[0x744419]
ruby(_ZN8rubinius16FinalizerHandler7performEPNS_5StateE+0x9b)[0x744e6b]
ruby(_ZN8rubinius28finalizer_handler_trampolineEPNS_5StateE+0x1c)[0x74504c]
ruby(_ZN8rubinius6Thread14execute_threadEPNS_5StateEPNS_2VME+0xbf)[0x720a0f]
ruby(_ZN8rubinius6Thread15internal_threadEPv+0x2f)[0x720aef]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a)[0x7f53e9572e9a]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f53e88663fd]
======= Memory map: ========
00400000-01577000 r-xp 00000000 ca:01 302755                             /usr/local/rvm/rubies/rbx-2.5.2/bin/rbx
01777000-017fd000 r--p 01177000 ca:01 302755                             /usr/local/rvm/rubies/rbx-2.5.2/bin/rbx
017fd000-01816000 rw-p 011fd000 ca:01 302755                             /usr/local/rvm/rubies/rbx-2.5.2/bin/rbx
01816000-01822000 rw-p 00000000 00:00 0 
01a61000-02815000 rw-p 00000000 00:00 0                                  [heap]
7f53b8000000-7f53b8021000 rw-p 00000000 00:00 0 
7f53b8021000-7f53bc000000 ---p 00000000 00:00 0 
7f53bf7ff000-7f53bf800000 ---p 00000000 00:00 0 
7f53bf800000-7f53c0000000 rw-p 00000000 00:00 0 
7f53c0000000-7f53c0021000 rw-p 00000000 00:00 0 
7f53c0021000-7f53c4000000 ---p 00000000 00:00 0 
7f53c4427000-7f53c4428000 ---p 00000000 00:00 0 
7f53c4428000-7f53c5628000 rw-p 00000000 00:00 0 
7f53c5628000-7f53c564d000 r-xp 00000000 ca:01 303986                     /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/psych-2.0.10/lib/psych.so
7f53c564d000-7f53c584c000 ---p 00025000 ca:01 303986                     /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/psych-2.0.10/lib/psych.so
7f53c584c000-7f53c584d000 r--p 00024000 ca:01 303986                     /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/psych-2.0.10/lib/psych.so
7f53c584d000-7f53c584e000 rw-p 00025000 ca:01 303986                     /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/psych-2.0.10/lib/psych.so
7f53c584e000-7f53c5855000 r-xp 00000000 ca:01 426637                     /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/json-1.8.2/lib/json/ext/generator.so
7f53c5855000-7f53c5a54000 ---p 00007000 ca:01 426637                     /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/json-1.8.2/lib/json/ext/generator.so
7f53c5a54000-7f53c5a55000 r--p 00006000 ca:01 426637                     /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/json-1.8.2/lib/json/ext/generator.so
7f53c5a55000-7f53c5a56000 rw-p 00007000 ca:01 426637                     /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/json-1.8.2/lib/json/ext/generator.so
7f53c5a56000-7f53c5a5b00D, [2015-02-06T22:25:27.540512 #18988] DEBUG -- : {"server_socket":"sending zy 0.0 json (flags=2)"}
0 r-xp 00000000 ca:01 426635                     /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/json-1.8.2/lib/json/ext/parser.so
7f53c5a5b000-7f53c5c5b000 ---p 00005000 ca:01 426635                     /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/json-1.8.2/lib/json/ext/parser.so
7f53c5c5b000-7f53c5c5c000 r--p 00005000 ca:01 426635                     /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/json-1.8.2/lib/json/ext/parser.so
7f53c5c5c000-7f53c5c5d000 rw-p 00006000 ca:01 426635                     /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/json-1.8.2/lib/json/ext/parser.so
7f53c5c5d000-7f53c5cad000 r-xp 00000000 ca:01 62992                      /usr/local/lib/libsodium.so.13.1.0
7f53c5cad000-7f53c5ead000 ---p 00050000 ca:01 62992                      /usr/local/lib/libsodium.so.13.1.0
7f53c5ead000-7f53c5eae000 r--p 00050000 ca:01 62992                      /usr/local/lib/libsodium.so.13.1.0
7f53c5eae000-7f53c5eb7000 rw-p 00051000 ca:01 62992                      /usr/local/lib/libsodium.so.13.1.0
7f53c5eb7000-7f53c5f1d000 r-xp 00000000 ca:01 63000                      /usr/local/lib/libzmq.so.4.0.0
7f53c5f1d000-7f53c611d000 ---p 00066000 ca:01 63000                      /usr/local/lib/libzmq.so.4.0.0
7f53c611d000-7f53c6121000 r--p 00066000 ca:01 63000                      /usr/local/lib/libzmq.so.4.0.0
7f53c6121000-7f53c6122000 rw-p 0006a000 ca:01 63000                      /usr/local/lib/libzmq.so.4.0.0
7f53c6122000-7f53c6129000 r-xp 00000000 ca:01 7717                       /usr/lib/x86_64-linux-gnu/libffi.so.6.0.0
7f53c6129000-7f53c6328000 ---p 00007000 ca:01 7717                       /usr/lib/x86_64-linux-gnu/libffi.so.6.0.0
7f53c6328000-7f53c6329000 r--p 00006000 ca:01 7717                       /usr/lib/x86_64-linux-gnu/libffi.so.6.0.0
7f53c6329000-7f53c632a000 rw-p 00007000 ca:01 7717                       /usr/lib/x86_64-linux-gnu/libffi.so.6.0.0
7f53c632a000-7f53c6348000 r-xp 00000000 ca:01 429370                     /usr/local/rvm/gems/rbx-2.5.2/gems/ffi-1.9.6/lib/ffi_c.so
7f53c6348000-7f53c6548000 ---p 0001e000 ca:01 429370                     /usr/local/rvm/gems/rbx-2.5.2/gems/ffi-1.9.6/lib/ffi_c.so
7f53c6548000-7f53c6549000 r--p 0001e000 ca:01 429370                     /usr/local/rvm/gems/rbx-2.5.2/gems/ffi-1.9.6/lib/ffi_c.so
7f53c6549000-7f53c654a000 rw-p 0001f000 ca:01 429370                     /usr/local/rvm/gems/rbx-2.5.2/gems/ffi-1.9.6/lib/ffi_c.so
7f53c654a000-7f53c659e000 r-xp 00000000 ca:01 135559                     /lib/x86_64-linux-gnu/libssl.so.1.0.0
7f53c659e000-7f53c679e000 ---p 00054000 ca:01 135559                     /lib/x86_64-linux-gnu/libssl.so.1.0.0
7f53c679e000-7f53c67a1000 r--p 00054000 ca:01 135559                     /lib/x86_64-linux-gnu/libssl.so.1.0.0
7f53c67a1000-7f53c67a7000 rw-p 00057000 ca:01 135559                     /lib/x86_64-linux-gnu/libssl.so.1.0.0
7f53c67a7000-7f53c67a8000 rw-p 00000000 00:00 0 
7f53c67a8000-7f53c67f4000 r-xp 00000000 ca:01 426025                     /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/rubysl-openssl-2.2.1/lib/openssl/openssl.so
7f53c67f4000-7f5D, [2015-02-06T22:25:27.540904 #18988] DEBUG -- : {"server_socket":"sent (11)"}
3c69f4000 ---p 0004c000 ca:01 426025                     /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/rubysl-openssl-2.2.1/lib/openssl/openssl.so
7f53c69f4000-7f53c69f5000 r--p 0004c000 ca:01 426025                     /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/rubysl-openssl-2.2.1/lib/openssl/openssl.so
7f53c69f5000-7f53c69f7000 rw-p 0004d000 ca:01 426025                     /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/rubysl-openssl-2.2.1/lib/openssl/openssl.so
7f53c69f7000-7f53c69f8000 rw-p 00000000 00:00 0 
7f53c69f8000-7f53c6a04000 r-xp 00000000 ca:01 427566                     /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/rubysl-zlib-2.0.1/lib/zlib/zlib.so
7f53c6a04000-7f53c6c03000 ---p 0000c000 ca:01 427566                     /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/rubysl-zlib-2.0.1/lib/zlib/zlib.so
7f53c6c03000-7f53c6c04000 r--p 0000b000 ca:01 427566                     /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/rubysl-zlib-2.0.1/lib/zlib/zlib.so
7f53c6c04000-7f53c6c05000 rw-p 0000c000 ca:01 427566            D, [2015-02-06T22:25:27.541114 #18988] DEBUG -- : {"server_socket":"sending {\"status\":[\"error\",\"request\",\"unroutable\"]} (flags=0)"}
         /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/rubysl-zlib-2.0.1/lib/zlib/zlib.so
7f53c6c05000-7f53c760d000 rw-p 00000000 00:00 0 
7f53c760d000-7f53c760e000 ---p 00000000 00:00 0 
7f53c760e000-7f53c7a0e000 rw-p 00000000 00:00 0 
7f53c7a0e000-7f53c7a24000 r-xp 00000000 ca:01 134871                     /lib/x86_64-linux-gnu/libz.so.1.2.3.4
7f53c7a24000-7f53c7c23000 ---p 00016000 ca:01 134871                     /lib/x86_64-linux-gnu/libz.so.1.2.3.4
7f53c7c23000-7f53c7c24000 r--p 00015000 ca:01 134871                     /lib/x86_64-linux-gnu/libz.so.1.2.3.4
7f53c7c24000-7f53c7c25000 rw-p 00016000 ca:01 134871                     /lib/x86_64-linux-gnu/libz.so.1.2.3.4
7f53c7c25000-7f53c7dd6000 r-xp 00000000 ca:01 135919                     /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7f53c7dd6000-7f53c7fd6000 ---p 001b1000 ca:01 135919                     /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7f53c7fd6000-7f53c7ff1000 r--p 001b1000 ca:01 135919                     /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7f53c7ff1000-7f53c7ffc000 rw-p 001cc000 ca:01 135919                     /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7f53c7ffc000-7f53c8000000 rw-p 00000000 00:00 0 
7f53c8000000-7f53c9810000 rw-p 00000000 00:00 0 
7f53c9810000-7f53cc000000 ---p 00000000 00:00 0 
7f53cc000000-7f53cc021000 rw-p 00000000 00:00 0 
7f53cc021000-7f53d0000000 ---p 00000000 00:00 0 
7f53d0000000-7f53d0021000 rw-p 00000000 00:00 0 
7f53d0021000-7f53d4000000 ---p 00000000 00:00 0 
7f53d4000000-7f53d4021000 rw-p 00000000 00:00 0 
7f53d4021000-7f53d8000000 ---p 00000000 00:00 0 
7f53d8000000-7f53d8021000 rw-p 00000000 00:00 0 
7f53d8021000-7f53dc000000 ---p 00000000 00:00 0 
7f53dc000000-7f53dc021000 rw-p 00000000 00:00 0 
7f53dc021000-7f53e0000000 ---p 00000000 00:00 0 
7f53e0000000-7f53e0021000 rw-p 00000000 00:00 0 
7f53e0021000-7f53e4000000 ---p 00000000 00:00 0 
7f53e4145000-7f53e4146000 r-xp 00000000 ca:01 66869                      /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/rubysl-digest-2.0.3/lib/digest/sha1/sha1.so
7f53e4146000-7f53e4345000 ---p 00001000 ca:01 66869                      /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/rubysl-digest-2.0.3/lib/digest/sha1/sha1.so
7f53e4345000-7f53e4346000 r--p 00000000 ca:01 66869                      /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/rubysl-digest-2.0.3/lib/digest/sha1/sha1.so
7f53e4346000-7f53e4347000 rw-p 00001000 ca:01 66869                      /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/rubysl-digest-2.0.3/lib/digest/sha1/sha1.so
7f53e4347000-7f53e4587000 rw-p 00000000 00:00 0 
7f53e4587000-7f53e4648000 ---p 00000000 00:00 0 
7f53e4648000-7f53e5473000 rw-p 00000000 00:00 0 
7f53e5473000-7f53e54a6000 r-xp 00000000 ca:01 303179                     /usr/local/rvm/rubies/rbx-2.5.2/runtime/gems/rubinius-melbourne-2.3.1.0/lib/rubinius/melbourne/melbourne.so
7f53e54a6000-7f53e56a5000 ---p 00033000 ca:01 303179                     /usr/local/rvm/rubies/rbx-2.5.2/runtime/gems/rubinius-melbourne-2.3.1.0/lib/rubinius/melbourne/melbourne.so
7f53e56a5000-7f53e56a7000 r--p 00032000 ca:01 303179                     /usr/local/rvm/rubies/rbx-2.5.2/runtime/gems/rubinius-melbourne-2.3.1.0/lib/rubinius/melbourne/melbourne.so
7f53e56a7000-7f53e56a8000 rw-p 00034000 ca:01 303179                     /usr/local/rvm/rubies/rbx-2.5.2/runtime/gems/rubinius-melbourne-2.3.1.0/lib/rubinius/melbourne/melbourne.so
7f53e56a8000-7f53e56a9000 ---p 00000000 00:00 0 
7f53e56a9000-7f53e5aa9000 rw-p 00000000 00:00 0 
7f53e5aa9000-7f53e5aaa000 ---p 00000000 00:00 0 
7f53e5aaa000-7f53e5eaa000 rw-p 00000000 00:00 0 
7f53e5eaa000-7f53e5eab000 ---p 00000000 00:00 0 
7f53e5eab000-7f53e62ab000 rw-p 00000000 00:00 0 
7f53e62ab000-7f53e62ac000 ---p 00000000 00:00 0 
7f53e62ac000-7f53e66ac000 rw-p 00000000 00:00 0 
7f53e66ac000-7f53e672c000 rwxp 00000000 00:00 0 
7f53e672c000-7f53e672d000 ---p 00000000 00:00 0 
7f53e672d000-7f53e6b2d000 rw-p 00000000 00:00 0 
7f53e6b2d000-7f53e6b2e000 ---p 00000000 00:00 0 
7f53e6b2e000-7f53e7936000 rw-p 00000000 00:00 0 
7f53e7b0c000-7f53e7b0d000 r-xp 00000000 00:00 0 
7f53e7b0d000-7f53e7b10000 r-xp 00000000 ca:01 66874                      /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/rubysl-digest-2.0.3/lib/digest/digest.so
7f53e7b10000-7f53e7d0f000 ---p 00003000 ca:01 66874                      /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/rubysl-digest-2.0.3/lib/digest/digest.so
7f53e7d0f000-7f53e7d10000 r--p 00002000 ca:01 66874                      /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/rubysl-digest-2.0.3/lib/digest/digest.so
7f53e7d10000-7f53e7d11000 rw-p 00003000 ca:01 66874                      /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/rubysl-digest-2.0.3/lib/digest/digest.so
7f53e7d11000-7f53e7d14000 r-xp 00000000 ca:01 427286                     /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/rubysl-io-console-2.0.0/lib/io/console/console.so
7f53e7d14000-7f53e7f13000 ---p 00003000 ca:01 427286                     /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/rubysl-io-console-2.0.0/lib/io/console/console.so
7f53e7f13000-7f53e7f14000 r--p 00002000 ca:01 427286                     /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/rubysl-io-console-2.0.0/lib/io/console/console.so
7f53e7f14000-7f53e7f15000 rw-p 00003000 ca:01 427286                     /usr/local/rvm/rubies/rbx-2.5.2/gems/gems/rubysl-io-console-2.0.0/lib/io/console/console.so
7f53e7f15000-7f53e7f36000 rw-p 00000000 00:00 0 
7f53e7f36000-7f53e7f42000 r-xp 00000000 ca:01 134840                     /lib/x86_64-linux-gnu/libnss_files-2.15.so
7f53e7f42000-7f53e8141000 ---p 0000c000 ca:01 134840                     /lib/x86_64-linux-gnu/libnss_files-2.15.so
7f53e8141000-7f53e8142000 r--p 0000b000 ca:01 134840                     /lib/x86_64-linux-gnu/libnss_files-2.15.so
7f53e8142000-7f53e8143000 rw-p 0000c000 ca:01 134840                     /lib/x86_64-linux-gnu/libnss_files-2.15.so
7f53e8143000-7f53e814d000 r-xp 00000000 ca:01 134842                     /lib/x86_64-linux-gnu/libnss_nis-2.15.so
7f53e814d000-7f53e834d000 ---p 0000a000 ca:01 134842                     /lib/x86_64-linux-gnu/libnss_nis-2.15.so
7f53e834d000-7f53e834e000 r--p 0000a000 ca:01 134842                     /lib/x86_64-linux-gnu/libnss_nis-2.15.so
7f53e834e000-7f53e834f000 rw-p 0000b000 ca:01 134842                     /lib/x86_64-linux-gnu/libnss_nis-2.15.so
7f53e834f000-7f53e8366000 r-xp 00000000 ca:01 134837                     /lib/x86_64-linux-gnu/libnsl-2.15.so
7f53e8366000-7f53e8565000 ---p 00017000 ca:01 134837                     /lib/x86_64-linux-gnu/libnsl-2.15.so
7f53e8565000-7f53e8566000 r--p 00016000 ca:01 134837                     /lib/x86_64-linux-gnu/libnsl-2.15.so
7f53e8566000-7f53e8567000 rw-p 00017000 ca:01 134837                     /lib/x86_64-linux-gnu/libnsl-2.15.so
7f53e8567000-7f53e8569000 rw-p 00000000 00:00 0 
7f53e8569000-7f53e8571000 r-xp 00000000 ca:01 134838                     /lib/x86_64-linux-gnu/libnss_compat-2.15.so
7f53e8571000-7f53e8770000 ---p 00008000 ca:01 134838                     /lib/x86_64-linux-gnu/libnss_compat-2.15.so
7f53e8770000-7f53e8771000 r--p 00007000 ca:01 134838                     /lib/x86_64-linux-gnu/libnss_compat-2.15.so
7f53e8771000-7D, [2015-02-06T22:25:27.542033 #18988] DEBUG -- : {"server_socket":"sent (43)"}
f53e8772000 rw-p 00008000 ca:01 134838                     /lib/x86_64-linux-gnu/libnss_compat-2.15.so
7f53e8772000-7f53e8927000 r-xp 00000000 ca:01 134814                     /lib/x86_64-linux-gnu/libc-2.15.so
7f53e8927000-7f53e8b27000 ---p 001b5000 ca:01 134814                     /lib/x86_64-linux-gnu/libc-2.15.so
7f53e8b27000-7f53e8b2b000 r--p 001b5000 ca:01 134814                     /lib/x86_64-linux-gnu/libc-2.15.so
7f53e8b2b000-7f53e8b2d000 rw-p 001b9000 ca:01 134814                     /lib/x86_64-linux-gnu/libc-2.15.so
7f53e8b2d000-7f53e8b32000 rw-p 00000000 00:00 0 
7f53e8b32000-7f53e8b47000 r-xp 00000000 ca:01 134824                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7f53e8b47000-7f53e8d46000 ---p 00015000 ca:01 134824                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7f53e8d46000-7f53e8d47000 r--p 00014000 ca:01 134824                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7f53e8d47000-7f53e8d48000 rw-p 00015000 ca:01 134824                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7f53e8d48000-7f53e8e2a000 r-xp 00000000 ca:01 7761                       /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.16
7f53e8e2a000-7f53e9029000 ---p 000e2000 ca:01 7761                       /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.16
7f53e9029000-7f53e9031000 r--p 000e1000 ca:01 7761                       /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.16
7f53e9031000-7f53e9033000 rw-p 000e9000 ca:01 7761                       /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.16
7f53e9033000-7f53e9048000 rw-p 00000000 00:00 0 
7f53e9048000-7f53e9143000 r-xp 00000000 ca:01 134830                     /lib/x86_64-linux-gnu/libm-2.15.so
7f53e9143000-7f53e9342000 ---p 000fb000 ca:01 134830                     /lib/x86_64-linux-gnu/libm-2.15.so
7f53e9342000-7f53e9343000 r--p 000fa000 ca:01 134830                     /lib/x86_64-linux-gnu/libm-2.15.so
7f53e9343000-7f53e9344000 rw-p 000fb000 ca:01 134830                     /lib/x86_64-linux-gnu/libm-2.15.so
7f53e9344000-7f53e9366000 r-xp 00000000 ca:01 134864                   D, [2015-02-06T22:25:27.542244 #18988] DEBUG -- : {"server_socket":"ready to recv"}
  /lib/x86_64-linux-gnu/libtinfo.so.5.9
7f53e9366000-7f53e9566000 ---p 00022000 ca:01 134864                     /lib/x86_64-linux-gnu/libtinfo.so.5.9
7f53e9566000-7f53e956a000 r--p 00022000 ca:01 134864                     /lib/x86_64-linux-gnu/libtinfo.so.5.9
7f53e956a000-7f53e956b000 rw-p 00026000 ca:01 134864                     /lib/x86_64-linux-gnu/libtinfo.so.5.9
7f53e956b000-7f53e9583000 r-xp 00000000 ca:01 134855                     /lib/x86_64-linux-gnu/libpthread-2.15.so
7f53e9583000-7f53e9782000 ---p 00018000 ca:01 134855                     /lib/x86_64-linux-gnu/libpthread-2.15.so
7f53e9782000-7f53e9783000 r--p 00017000 ca:01 134855                     /lib/x86_64-linux-gnu/libpthread-2.15.so
7f53e9783000-7f53e9784000 rw-p 00018000 ca:01 134855                     /lib/x86_64-linux-gnu/libpthread-2.15.so
7f53e9784000-7f53e9788000 rw-p 00000000 00:00 0 
7f53e9788000-7f53e978a000 r-xp 00000000 ca:01 134820                     /lib/x86_64-linux-gnu/libdl-2.15.so
7f53e978a000-7f53e998a000 ---p 00002000 ca:01 134820                     /lib/x86_64-linux-gnu/libdl-2.15.so
7f53e998a000-7f53e998b000 r--p 00002000 ca:01 134820                     /lib/x86_64-linux-gnu/libdl-2.15.so
7f53e998b000-7f53e998c000 rw-p 00003000 ca:01 134820                     /lib/x86_64-linux-gnu/libdl-2.15.so
7f53e998c000-7f53e9995000 r-xp 00000000 ca:01 134817                     /lib/x86_64-linux-gnu/libcrypt-2.15.so
7f53e9995000-7f53e9b95000 ---p 00009000 ca:01 134817                     /lib/x86_64-linux-gnu/libcrypt-2.15.so
7f53e9b95000-7f53e9b96000 r--p 00009000 ca:01 134817                     /lib/x86_64-linux-gnu/libcrypt-2.15.so
7f53e9b96000-7f53e9b97000 rw-p 0000a000 ca:01 134817                     /lib/x86_64-linux-gnu/libcrypt-2.15.so
7f53e9b97000-7f53e9bc5000 rw-p 00000000 00:00 0 
7f53e9bc5000-7f53e9bcc000 r-xp 00000000 ca:01 134858                     /lib/x86_64-linux-gnu/librt-2.15.so
7f53e9bcc000-7f53e9dcb000 ---p 00007000 ca:01 134858                     /lib/x86_64-linux-gnu/librt-2.15.so
7f53e9dcb000-7f53e9dcc000 r--p 00006000 ca:01 134858                     /lib/x86_64-linux-gnu/librt-2.15.so
7f53e9dcc000-7f53e9dcd000 rw-p 00007000 ca:01 134858                     /lib/x86_64-linux-gnu/librt-2.15.so
7f53e9dcd000-7f53e9def000 r-xp 00000000 ca:01 134805                     /lib/x86_64-linux-gnu/ld-2.15.so
7f53e9dfd000-7f53e9f86000 r--p 00000000 ca:01 27221                      /usr/lib/locale/locale-archive
7f53e9f86000-7f53e9fe1000 rw-p 00000000 00:00 0 
7f53e9fea000-7f53e9feb000 rw-p 00000000 00:00 0 
7f53e9feb000-7f53e9fed000 rwxp 00000000 00:00 0 
7f53e9fed000-7f53e9fef000 rw-p 00000000 00:00 0 
7f53e9fef000-7f53e9ff0000 r--p 00022000 ca:01 134805                     /lib/x86_64-linux-gnu/ld-2.15.so
7f53e9ff0000-7f53e9ff2000 rw-p 00023000 ca:01 134805                     /lib/x86_64-linux-gnu/ld-2.15.so
7fffac947000-7fffac99b000 rw-p 00000000 00:00 0                          [stack]
7fffac9ff000-7fffaca00000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
The Rubinius process is aborting with signal: SIGABRT
--- begin system info ---
sysname: Linux
nodename: levis-dev-app-0
release: 3.2.0-59-virtual
version: #90-Ubuntu SMP Tue Jan 7 23:02:33 UTC 2014
machine: x86_64
--- end system info ---
--- begin rubinius info ---
program name: rbx
version: 2.5.2
ruby version: 2.1.0
release date: 2015-01-30
build revision: 7a5b05b1a9aca87703ceb0bafe00b916b9510285
llvm version: 3.4
--- end rubinius info ---
--- begin system backtrace ---
ruby() [0x6980da]
/lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f53e957acb0]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f53e87a8425]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7f53e87abb8b]
/lib/x86_64-linux-gnu/libc.so.6(+0x7439e) [0x7f53e87e639e]
/lib/x86_64-linux-gnu/libc.so.6(+0x7eb96) [0x7f53e87f0b96]
/usr/local/rvm/gems/rbx-2.5.2/gems/ffi-1.9.6/lib/ffi_c.so(+0x1537e) [0x7f53c633f37e]
ruby(_ZN8rubinius16FinalizerHandler8finalizeEPNS_5StateE+0x229) [0x744419]
ruby(_ZN8rubinius16FinalizerHandler7performEPNS_5StateE+0x9b) [0x744e6b]
ruby(_ZN8rubinius28finalizer_handler_trampolineEPNS_5StateE+0x1c) [0x74504c]
ruby(_ZN8rubinius6Thread14execute_threadEPNS_5StateEPNS_2VME+0xbf) [0x720a0f]
ruby(_ZN8rubinius6Thread15internal_threadEPv+0x2f) [0x720aef]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a) [0x7f53e9572e9a]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f53e88663fd]
--- end system backtrace ---
--- begin Ruby backtraces ---
--- Thread 1 backtrace ---
0x7fffac993a90: capi:zmq_recvmsg at Function.c
ZMQ::Socket#__recvmsg__ in /usr/local/rvm/gems/rbx-2.5.2/gems/ffi-rzmq-2.0.4/lib/ffi-rzmq/socket.rb:584 (+11)
ZMQ::Socket#recvmsg in /usr/local/rvm/gems/rbx-2.5.2/gems/ffi-rzmq-2.0.4/lib/ffi-rzmq/socket.rb:321 (+22)
__block__ in /usr/local/rvm/gems/rbx-2.5.2/bundler/gems/zy-5fabfce565ec/lib/zy/server.rb:60 (+81)
Kernel#loop in kernel/common/kernel.rb:510 (+27)
Zy::Server#start in /usr/local/rvm/gems/rbx-2.5.2/bundler/gems/zy-5fabfce565ec/lib/zy/server.rb:54 (+684)
Object#__script__ in /app/venue/bin/venue.rb:16 (+226)
Rubinius::CodeLoader#load_script in kernel/delta/code_loader.rb:66 (+52)
Rubinius::CodeLoader.load_script in kernel/delta/code_loader.rb:152 (+40)
Rubinius::Loader#script in kernel/loader.rb:655 (+214)
Rubinius::Loader#main in kernel/loader.rb:809 (+77)
--- Thread 2 backtrace ---
Segmentation fault (core dumped)
notEthan commented 9 years ago

interestingly, the server did actually respond that time - it got the two request frames and sent two reply frames (even though they are not logged); a client running on ruby 2.0.0 completed a successful request and reply after which the server (rbx) segfaulted.

chuckremes commented 9 years ago

Okay, I’ll take a look at your client & server code tomorrow to see if I can reproduce. Might be a bug in ffi-rzmq… we’ll see.

On Feb 6, 2015, at 4:27 PM, notEthan notifications@github.com wrote:

interestingly, the server did actually respond that time - it got the two request frames and sent two reply frames (even though they are not logged); a client running on ruby 2.0.0 completed a successful request and reply after which the server segfaulted.

— Reply to this email directly or view it on GitHub https://github.com/chuckremes/ffi-rzmq/issues/118#issuecomment-73324632.

notEthan commented 9 years ago

thanks, I appreciate it. also I ran on rbx on my mac osx machine and the client and server operate fine. just the ubuntu box seems to have issues.

chuckremes commented 9 years ago

What version of zeromq are you running?

I’m running OSX too, so I assume it will work on my box. I don’t really have an Ubuntu box or VM available right now (funny, I would have one in about a week). Anyway, give me all the details so I can try to repro.

what version of ffi-rzmq? what version of zeromq? which ubuntu?

Thanks

On Feb 6, 2015, at 5:00 PM, notEthan notifications@github.com wrote:

thanks, I appreciate it. also I ran on rbx on my mac osx machine and the client and server operate fine. just the ubuntu box seems to have issues.

— Reply to this email directly or view it on GitHub https://github.com/chuckremes/ffi-rzmq/issues/118#issuecomment-73329178.

notEthan commented 9 years ago

these were all built or installed just today (following https://github.com/zeromq/libcurve#building-and-installing ), so should be the latest versions. ubuntu is not, though. ubuntu 12.04 precise ffi-rzmq (2.0.4) ffi-rzmq-core (1.0.3) LibZMQ.version => {:major=>4, :minor=>2, :patch=>0}

it occurs to me that, since those instructions clone the used libraries straight from master, it may not be a stable version, and that I should go to an actual release version, particularly of libzmq. I will try that, probably early next week.

also I will be trying on an ubuntu 14.04 box early next week to see if it behaves any differently.

the code linked that's used by the server (zy, at https://github.com/notEthan/zy) isn't quite enough to run on its own, it is a framework without an actual application. if you wish to try running it, you can clone the repo and run it with a stub application, doing something like:

ruby -I /path/to/zy/lib -r zy -e "Zy::Server.new(:app => proc { {} }, :bind => 'tcp://*:5113', 'server_private_key' => 'a private key here').start"

or in an executable

#!/usr/bin/env ruby

$: << '/path/to/zy/lib'

require 'zy'

server_private_key = # a private key here
Zy::Server.new(:app => proc { {} }, :bind => 'tcp://*:5113', 'server_private_key' => server_private_key).start

the :server_private_key argument can be omitted. the segfaults seem to behave the same with or without curve, so probably simpler to omit.

the client (yz, at https://github.com/notEthan/yz) can be run in irb with

$: << '/path/to/yz/lib'
require 'yz'
client = Yz::Client.new(:connect => 'tcp://localhost:5113')
# OR 
client = Yz::Client.new(:connect => 'tcp://localhost:5113', server_public_key: "a key", client_public_key: "a key", client_private_key: "a key")

client.request({})

let me know if I can provide anything else.

notEthan commented 9 years ago

things seem to work all right on ubuntu 14.04 with an actual libzmq release (both 4.0.5 stable and 4.1.0 release candidate)

as I mentioned above, I realized I was installing libzmq from git master which was potentially unstable, and did seem to be the source of the problem, both on ubuntu 12.04 and 14.04 (consistent segfaults).

there are still problems with the release versions (4.0.5 and 4.1.0) on ubuntu 12.04 - with both, the service just hangs rather than accepting any connections, but also when building, make check hangs on one particular test. this seems clear to not be a ffi-rzmq issue.

sorry for the noise and not realizing earlier on I wasn't on a stable release.

chuckremes commented 9 years ago

I’m glad you found a reasonable resolution. Tracking down a problem where a Ruby binding is causing a C lib to fault is a really tall order. Phew!

On Feb 10, 2015, at 19:45, notEthan notifications@github.com wrote:

things seem to work all right on ubuntu 14.04 with an actual libzmq release (both 4.0.5 stable and 4.1.0 release candidate)

as I mentioned above, I realized I was installing libzmq from git master which was potentially unstable, and did seem to be the source of the problem, both on ubuntu 12.04 and 14.04 (consistent segfaults).

there are still problems with the release versions (4.0.5 and 4.1.0) on ubuntu 12.04 - with both, the service just hangs rather than accepting any connections, but also when building, make check hangs on one particular test. this seems clear to not be a ffi-rzmq issue.

sorry for the noise and not realizing earlier on I wasn't on a stable release.

— Reply to this email directly or view it on GitHub https://github.com/chuckremes/ffi-rzmq/issues/118#issuecomment-73821636.