Vasfed / heap_dump

ruby gem for dumping heap references
MIT License
91 stars 13 forks source link

Segfault in a large Rails 2.3.x app on Ruby 1.9.3p125 (2012-02-16 revision 34643) [x86_64-linux] #2

Closed mrb closed 12 years ago

mrb commented 12 years ago

Very excited about this project! I know you don't fully support 1.9.3 yet, but maybe this is a relatively minor fix?

/opt/src/paperlesspost/heap_dump/lib/heap_dump.rb:10: [BUG] Segmentation fault
ruby 1.9.3p125 (2012-02-16 revision 34643) [x86_64-linux]

-- Control frame information -----------------------------------------------
c:0031 p:---- s:0179 b:0179 l:000178 d:000178 CFUNC  :dump_ext
c:0030 p:0041 s:0175 b:0175 l:000174 d:000174 METHOD /opt/src/paperlesspost/heap_dump/lib/heap_dump.rb:10
c:0029 p:0020 s:0170 b:0170 l:000169 d:000169 METHOD /opt/src/paperlesspost/paperless-post/config/initializers/heap_dump.rb:12
c:0028 p:0106 s:0166 b:0166 l:000165 d:000165 METHOD /opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/sass-3.2.1/lib/sass/plugin/rack.rb:54
c:0027 p:0015 s:0162 b:0162 l:000161 d:000161 METHOD /opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/actionpack-2.3.14.pped08b10/lib/action_controller/string_co
c:0026 p:0015 s:0155 b:0155 l:000154 d:000154 METHOD /opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/rack-1.1.3/lib/rack/head.rb:9
c:0025 p:0155 s:0148 b:0148 l:000147 d:000147 METHOD /opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/rack-1.1.3/lib/rack/methodoverride.rb:24
c:0024 p:0046 s:0142 b:0142 l:000141 d:000141 METHOD /opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/actionpack-2.3.14.pped08b10/lib/action_controller/params_pa
c:0023 p:0027 s:0137 b:0137 l:000136 d:000136 METHOD /opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/actionpack-2.3.14.pped08b10/lib/action_controller/session/a
c:0022 p:0017 s:0126 b:0126 l:000125 d:000125 METHOD /opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/airbrake-3.1.2/lib/airbrake/rack.rb:42
c:0021 p:0015 s:0120 b:0120 l:000119 d:000119 METHOD /opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/actionpack-2.3.14.pped08b10/lib/action_controller/failsafe.
c:0020 p:0015 s:0115 b:0115 l:000114 d:000114 METHOD /opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/airbrake-3.1.2/lib/airbrake/user_informer.rb:12
c:0019 p:0014 s:0106 b:0106 l:000100 d:000105 BLOCK  /opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/rack-1.1.3/lib/rack/lock.rb:11
c:0018 p:0019 s:0104 b:0104 l:000103 d:000103 METHOD <internal:prelude>:10
c:0017 p:0054 s:0101 b:0101 l:000100 d:000100 METHOD /opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/rack-1.1.3/lib/rack/lock.rb:11
c:0016 p:0024 s:0096 b:0096 l:000086 d:000095 BLOCK  /opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/actionpack-2.3.14.pped08b10/lib/action_controller/dispatche
c:0015 p:0040 s:0094 b:0094 l:000093 d:000093 METHOD /opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/actionpack-2.3.14.pped08b10/lib/action_controller/reloader.
c:0014 p:0038 s:0087 b:0087 l:000086 d:000086 METHOD /opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/actionpack-2.3.14.pped08b10/lib/action_controller/dispatche
c:0013 p:0193 s:0083 b:0083 l:000082 d:000082 METHOD /opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/rails-2.3.14.pped08b10/lib/rails/rack/static.rb:31
c:0012 p:0167 s:0076 b:0076 l:000063 d:000075 BLOCK  /opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/rack-1.1.3/lib/rack/urlmap.rb:47
c:0011 p:---- s:0069 b:0069 l:000068 d:000068 FINISH
c:0010 p:---- s:0067 b:0067 l:000066 d:000066 CFUNC  :each
c:0009 p:0064 s:0064 b:0064 l:000063 d:000063 METHOD /opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/rack-1.1.3/lib/rack/urlmap.rb:41
c:0008 p:0015 s:0055 b:0055 l:000054 d:000054 METHOD /opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/rails-2.3.14.pped08b10/lib/rails/rack/log_tailer.rb:17
c:0007 p:0015 s:0050 b:0050 l:000049 d:000049 METHOD /opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/rack-1.1.3/lib/rack/content_length.rb:13
c:0006 p:0331 s:0042 b:0042 l:000041 d:000041 METHOD /opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/rack-1.1.3/lib/rack/handler/webrick.rb:48
c:0005 p:0257 s:0030 b:0030 l:000029 d:000029 METHOD /usr/lib64/ruby/1.9.1/webrick/httpserver.rb:138
c:0004 p:0393 s:0020 b:0020 l:000019 d:000019 METHOD /usr/lib64/ruby/1.9.1/webrick/httpserver.rb:94
c:0003 p:0126 s:0009 b:0009 l:0013f8 d:000008 BLOCK  /usr/lib64/ruby/1.9.1/webrick/server.rb:191
c:0002 p:---- s:0004 b:0004 l:000003 d:000003 FINISH
c:0001 p:---- s:0002 b:0002 l:000001 d:000001 TOP   

-- Ruby level backtrace information ----------------------------------------
/usr/lib64/ruby/1.9.1/webrick/server.rb:191:in `block in start_thread'
/usr/lib64/ruby/1.9.1/webrick/httpserver.rb:94:in `run'
/usr/lib64/ruby/1.9.1/webrick/httpserver.rb:138:in `service'
/opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/rack-1.1.3/lib/rack/handler/webrick.rb:48:in `service'
/opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/rack-1.1.3/lib/rack/content_length.rb:13:in `call'
/opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/rails-2.3.14.pped08b10/lib/rails/rack/log_tailer.rb:17:in `call'
/opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/rack-1.1.3/lib/rack/urlmap.rb:41:in `call'
/opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/rack-1.1.3/lib/rack/urlmap.rb:41:in `each'
/opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/rack-1.1.3/lib/rack/urlmap.rb:47:in `block in call'
/opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/rails-2.3.14.pped08b10/lib/rails/rack/static.rb:31:in `call'
/opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/actionpack-2.3.14.pped08b10/lib/action_controller/dispatcher.rb:108:in `call'
/opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/actionpack-2.3.14.pped08b10/lib/action_controller/reloader.rb:34:in `run'
/opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/actionpack-2.3.14.pped08b10/lib/action_controller/dispatcher.rb:114:in `block in call'
/opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/rack-1.1.3/lib/rack/lock.rb:11:in `call'
<internal:prelude>:10:in `synchronize'
/opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/rack-1.1.3/lib/rack/lock.rb:11:in `block in call'
/opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/airbrake-3.1.2/lib/airbrake/user_informer.rb:12:in `call'
/opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/actionpack-2.3.14.pped08b10/lib/action_controller/failsafe.rb:26:in `call'
/opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/airbrake-3.1.2/lib/airbrake/rack.rb:42:in `call'
/opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/actionpack-2.3.14.pped08b10/lib/action_controller/session/abstract_store.rb:177:in `call'
/opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/actionpack-2.3.14.pped08b10/lib/action_controller/params_parser.rb:15:in `call'
/opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/rack-1.1.3/lib/rack/methodoverride.rb:24:in `call'
/opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/rack-1.1.3/lib/rack/head.rb:9:in `call'
/opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/actionpack-2.3.14.pped08b10/lib/action_controller/string_coercion.rb:25:in `call'
/opt/src/paperlesspost/paperless-post/vendor/bundle/ruby/1.9.1/gems/sass-3.2.1/lib/sass/plugin/rack.rb:54:in `call'
/opt/src/paperlesspost/paperless-post/config/initializers/heap_dump.rb:12:in `call'
/opt/src/paperlesspost/heap_dump/lib/heap_dump.rb:10:in `dump'
/opt/src/paperlesspost/heap_dump/lib/heap_dump.rb:10:in `dump_ext'

-- C level backtrace information -------------------------------------------
/usr/lib64/libruby.so.1.9(+0x150f22) [0x7fccaedabf22] vm_dump.c:796
/usr/lib64/libruby.so.1.9(+0x57566) [0x7fccaecb2566] error.c:258
/usr/lib64/libruby.so.1.9(rb_bug+0xb8) [0x7fccaecb26d8] error.c:277
/usr/lib64/libruby.so.1.9(+0xef075) [0x7fccaed4a075] signal.c:609
/lib64/libpthread.so.0(+0xf500) [0x7fccaea4d500]
/opt/src/paperlesspost/heap_dump/lib/heap_dump.so(+0x4840) [0x7fcca01dd840] ../../../../ext/heap_dump/heap_dump.c:1422
/opt/src/paperlesspost/heap_dump/lib/heap_dump.so(heapdump_dump+0xc1) [0x7fcca01dda01] ../../../../ext/heap_dump/heap_dump.c:1490
/opt/src/paperlesspost/heap_dump/lib/heap_dump.so(+0x4b06) [0x7fcca01ddb06] ../../../../ext/heap_dump/heap_dump.c:1536
/usr/lib64/libruby.so.1.9(+0x13ff1b) [0x7fccaed9af1b] vm_insnhelper.c:404
/usr/lib64/libruby.so.1.9(+0x142720) [0x7fccaed9d720] insns.def:1015
/usr/lib64/libruby.so.1.9(+0x147319) [0x7fccaeda2319] vm.c:1220
/usr/lib64/libruby.so.1.9(rb_yield+0x248) [0x7fccaeda9248] vm.c:624
/usr/lib64/libruby.so.1.9(rb_ary_each+0x57) [0x7fccaec85cd7] array.c:1478
/usr/lib64/libruby.so.1.9(+0x13ff1b) [0x7fccaed9af1b] vm_insnhelper.c:404
/usr/lib64/libruby.so.1.9(+0x142720) [0x7fccaed9d720] insns.def:1015
/usr/lib64/libruby.so.1.9(+0x147319) [0x7fccaeda2319] vm.c:1220
/usr/lib64/libruby.so.1.9(+0x148b03) [0x7fccaeda3b03] vm.c:624
/usr/lib64/libruby.so.1.9(+0x156d08) [0x7fccaedb1d08] thread.c:453
/usr/lib64/libruby.so.1.9(+0x156d60) [0x7fccaedb1d60] thread_pthread.c:653
/lib64/libpthread.so.0(+0x7851) [0x7fccaea45851]
/lib64/libc.so.6(clone+0x6d) [0x7fccadecc11d] regparse.c:3041

-- Other runtime information -----------------------------------------------
Vasfed commented 12 years ago

Does it crash every time or only occasionally? What's content of config/initializers/heap_dump.rb?

mrb commented 12 years ago

@Vasfed It crashes every time. I first caused a segfault while attaching with gdb, and then tried it in Ruby to see if there was any difference.

config/initializers/heap_dump.rb is just an empty class that calls HeapDump.dump

I can provide you with the full output for the crash in either context (gdb or within Rails) if you want, let me know.

Vasfed commented 12 years ago

Found some 1.9.3-specific segfault, but unsure if it's the cause of subj, please test against current head (if using bundler - gem 'heap_dump', git:'git://github.com/Vasfed/heap_dump.git', if not using - suggest doing so, http://gembundler.com/rails23.html)

mrb commented 12 years ago

@Vasfed You got it! Thanks a ton. Closing issue.

Vasfed commented 12 years ago

Released v0.0.27 with this fix

mrb commented 12 years ago

@Vasfed :+1: Awesome. I can't tell you how helpful this is going to be for me.

Vasfed commented 12 years ago

@mrb You're welcome. I haven't used it for rails yet, but had for cramp-based - had 2 actions in actual production app, one that runs dump, other - just counts instances of the app's namespace objects (some version of it may be coming in heapdump).

Feel free to share your ideas about making heapdump more handy

mrb commented 12 years ago

@Vasfed Awesome, will do. We'll likely add a protected/authenticated conditional route in our app that let's us run a heap dump when we need it.