search

sensu-plugins / sensu-plugins-disk-checks

This plugin provides native disk instrumentation for monitoring and metrics collection, including: health, usage, and various metrics.
http://sensu-plugins.io
MIT License
27 stars 63 forks source link

Segmentation fault on filesystem.rb after ruby update #65

Closed totojack closed 7 years ago

totojack commented 7 years ago

Hi.

We experienced a segmentation fault on check-disk-usage.rb on a centos 6.8 after ruby update. EMBEDDED_RUBY is enabled.

/opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/sys-filesystem-1.1.5/lib/sys/unix/sys/filesystem.rb:315: [BUG] Segmentation fault at 0x00000000000000 ruby 2.3.0p0 (2015-12-25 revision 53290) [x86_64-linux]

-- Control frame information ----------------------------------------------- c:0006 p:---- s:0026 e:000025 CFUNC :getmntent c:0005 p:0581 s:0022 e:000021 METHOD /opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/sys-filesystem-1.1.5/lib/sys/unix/sys/filesystem.rb:315 c:0004 p:0014 s:0012 e:000011 METHOD /opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/sensu-plugins-disk-checks-1.1.3/bin/check-disk-usage.rb:121 c:0003 p:0008 s:0009 e:000008 METHOD /opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/sensu-plugins-disk-checks-1.1.3/bin/check-disk-usage.rb:206 c:0002 p:0020 s:0006 e:000005 BLOCK /opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/sensu-plugin-1.2.0/lib/sensu-plugin/cli.rb:56 [FINISH] c:0001 p:0000 s:0002 E:002330 (none) [FINISH]

-- Ruby level backtrace information ---------------------------------------- /opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/sensu-plugin-1.2.0/lib/sensu-plugin/cli.rb:56:in block in <class:CLI>' /opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/sensu-plugins-disk-checks-1.1.3/bin/check-disk-usage.rb:206:inrun' /opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/sensu-plugins-disk-checks-1.1.3/bin/check-disk-usage.rb:121:in fs_mounts' /opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/sys-filesystem-1.1.5/lib/sys/unix/sys/filesystem.rb:315:inmounts' /opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/sys-filesystem-1.1.5/lib/sys/unix/sys/filesystem.rb:315:in `getmntent'

-- Machine register context ------------------------------------------------ RIP: 0x00007f2bc44fdceb RBP: 0x0000000000000000 RSP: 0x00007ffe33ea8760 RAX: 0x0000000000000000 RBX: 0x00000000013b4ce0 RCX: 0x0000000000001000 RDX: 0x00000000013b4ce0 RDI: 0x0000000000000000 RSI: 0x00007f2bc47ac060 R8: 0x00007f2bc47a9188 R9: 0x0000000000000001 R10: 0x0000000000001000 R11: 0x0000000000000206 R12: 0x00007f2bc47ac060 R13: 0x0000000000001000 R14: 0x00000000013843e8 R15: 0x00007f2bc598fed0 EFL: 0x0000000000010206

-- C level backtrace information ------------------------------------------- /opt/sensu/embedded/lib/libruby.so.2.3 [0x7f2bc547e91d] /opt/sensu/embedded/lib/libruby.so.2.3 [0x7f2bc547ed5a] /opt/sensu/embedded/lib/libruby.so.2.3 [0x7f2bc5302b9c] /opt/sensu/embedded/lib/libruby.so.2.3 [0x7f2bc53e1d2d] /lib64/libpthread.so.0 [0x7f2bc50857e0] /lib64/libc.so.6(__getmntent_r+0x1b) [0x7f2bc44fdceb] /opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/ffi-1.9.14/lib/ffi_c.so(ffi_call_unix64+0x4c) [0x7f2bbdb6d264] /opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/ffi-1.9.14/ext/ffi_c/libffi/src/x86/ffi64.c:633 /opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/ffi-1.9.14/lib/ffi_c.so(ffi_call+0x3e3) [0x7f2bbdb6ccc0] /opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/ffi-1.9.14/ext/ffi_c/libffi/src/x86/ffi64.c:486 /opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/ffi-1.9.14/lib/ffi_c.so(rbffi_CallFunction+0x3fc) [0x7f2bbdb6ba7f] /opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/ffi-1.9.14/lib/ffi_c.so [0x7f2bbdb5ffeb] /opt/sensu/embedded/lib/libruby.so.2.3 [0x7f2bc5462258] /opt/sensu/embedded/lib/libruby.so.2.3 [0x7f2bc5462d96] /opt/sensu/embedded/lib/libruby.so.2.3 [0x7f2bc5462f95] /opt/sensu/embedded/lib/libruby.so.2.3 [0x7f2bc5463d49] /opt/sensu/embedded/lib/libruby.so.2.3 [0x7f2bc546440a] /opt/sensu/embedded/lib/libruby.so.2.3 [0x7f2bc54645e5] /opt/sensu/embedded/lib/libruby.so.2.3 [0x7f2bc5468340] /opt/sensu/embedded/lib/libruby.so.2.3 [0x7f2bc547a135] /opt/sensu/embedded/lib/libruby.so.2.3 [0x7f2bc5478558] /opt/sensu/embedded/lib/libruby.so.2.3 [0x7f2bc5478414] /opt/sensu/embedded/lib/libruby.so.2.3 [0x7f2bc54785bb] /opt/sensu/embedded/lib/libruby.so.2.3 [0x7f2bc547885a] /opt/sensu/embedded/lib/libruby.so.2.3 [0x7f2bc5478987] /opt/sensu/embedded/lib/libruby.so.2.3 [0x7f2bc53101f5] /opt/sensu/embedded/lib/libruby.so.2.3 [0x7f2bc5308196] /opt/sensu/embedded/lib/libruby.so.2.3 [0x7f2bc5308339] /opt/sensu/embedded/lib/libruby.so.2.3 [0x7f2bc5308440] /opt/sensu/embedded/lib/libruby.so.2.3 [0x7f2bc53088b3] /opt/sensu/embedded/lib/libruby.so.2.3 [0x7f2bc5308b5e] /opt/sensu/embedded/lib/libruby.so.2.3 [0x7f2bc5309099] 00400000-00401000 r-xp 00000000 fd:00 26685 /opt/sensu/embedded/bin/ruby 00600000-00601000 rw-p 00000000 fd:00 26685 /opt/sensu/embedded/bin/ruby 00ef4000-01419000 rw-p 00000000 00:00 0 [heap] 7f2bbd765000-7f2bbd93b000 r--s 00000000 fd:00 2886 /lib64/libc-2.12.so 7f2bbd93b000-7f2bbd951000 r-xp 00000000 fd:00 2336 /lib64/libgcc_s-4.4.7-20120601.so.1 7f2bbd951000-7f2bbdb50000 ---p 00016000 fd:00 2336 /lib64/libgcc_s-4.4.7-20120601.so.1 7f2bbdb50000-7f2bbdb51000 rw-p 00015000 fd:00 2336 /lib64/libgcc_s-4.4.7-20120601.so.1 7f2bbdb51000-7f2bbdb75000 r-xp 00000000 fd:00 279276 /opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/ffi-1.9.14/lib/ffi_c.so 7f2bbdb75000-7f2bbdd74000 ---p 00024000 fd:00 279276 /opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/ffi-1.9.14/lib/ffi_c.so 7f2bbdd74000-7f2bbdd76000 rw-p 00023000 fd:00 279276 /opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/ffi-1.9.14/lib/ffi_c.so 7f2bbdd76000-7f2bbdd7e000 r-xp 00000000 fd:00 31288 /opt/sensu/embedded/lib/ruby/2.3.0/x86_64-linux/stringio.so 7f2bbdd7e000-7f2bbdf7d000 ---p 00008000 fd:00 31288 /opt/sensu/embedded/lib/ruby/2.3.0/x86_64-linux/stringio.so 7f2bbdf7d000-7f2bbdf7e000 rw-p 00007000 fd:00 31288 /opt/sensu/embedded/lib/ruby/2.3.0/x86_64-linux/stringio.so 7f2bbdf7e000-7f2bbdf80000 r-xp 00000000 fd:00 137371 /opt/sensu/embedded/lib/ruby/2.3.0/x86_64-linux/enc/trans/transdb.so 7f2bbdf80000-7f2bbe180000 ---p 00002000 fd:00 137371 /opt/sensu/embedded/lib/ruby/2.3.0/x86_64-linux/enc/trans/transdb.so 7f2bbe180000-7f2bbe181000 rw-p 00002000 fd:00 137371 /opt/sensu/embedded/lib/ruby/2.3.0/x86_64-linux/enc/trans/transdb.so 7f2bbe181000-7f2bbe183000 r-xp 00000000 fd:00 137329 /opt/sensu/embedded/lib/ruby/2.3.0/x86_64-linux/enc/encdb.so 7f2bbe183000-7f2bbe382000 ---p 00002000 fd:00 137329 /opt/sensu/embedded/lib/ruby/2.3.0/x86_64-linux/enc/encdb.so 7f2bbe382000-7f2bbe383000 rw-p 00001000 fd:00 137329 /opt/sensu/embedded/lib/ruby/2.3.0/x86_64-linux/enc/encdb.so 7f2bbe383000-7f2bc4218000 r--p 00000000 fd:00 15722 /usr/lib/locale/locale-archive 7f2bc4218000-7f2bc421a000 r-xp 00000000 fd:00 8981 /lib64/libfreebl3.so 7f2bc421a000-7f2bc4419000 ---p 00002000 fd:00 8981 /lib64/libfreebl3.so 7f2bc4419000-7f2bc441a000 r--p 00001000 fd:00 8981 /lib64/libfreebl3.so 7f2bc441a000-7f2bc441b000 rw-p 00002000 fd:00 8981 /lib64/libfreebl3.so 7f2bc441b000-7f2bc45a5000 r-xp 00000000 fd:00 2886 /lib64/libc-2.12.so 7f2bc45a5000-7f2bc47a5000 ---p 0018a000 fd:00 2886 /lib64/libc-2.12.so 7f2bc47a5000-7f2bc47a9000 r--p 0018a000 fd:00 2886 /lib64/libc-2.12.so 7f2bc47a9000-7f2bc47ab000 rw-p 0018e000 fd:00 2886 /lib64/libc-2.12.so 7f2bc47ab000-7f2bc47af000 rw-p 00000000 00:00 0 7f2bc47af000-7f2bc4832000 r-xp 00000000 fd:00 10760 /lib64/libm-2.12.so 7f2bc4832000-7f2bc4a31000 ---p 00083000 fd:00 10760 /lib64/libm-2.12.so 7f2bc4a31000-7f2bc4a32000 r--p 00082000 fd:00 10760 /lib64/libm-2.12.so 7f2bc4a32000-7f2bc4a33000 rw-p 00083000 fd:00 10760 /lib64/libm-2.12.so 7f2bc4a33000-7f2bc4a3a000 r-xp 00000000 fd:00 2890 /lib64/libcrypt-2.12.so 7f2bc4a3a000-7f2bc4c3a000 ---p 00007000 fd:00 2890 /lib64/libcrypt-2.12.so 7f2bc4c3a000-7f2bc4c3b000 r--p 00007000 fd:00 2890 /lib64/libcrypt-2.12.so 7f2bc4c3b000-7f2bc4c3c000 rw-p 00008000 fd:00 2890 /lib64/libcrypt-2.12.so 7f2bc4c3c000-7f2bc4c6a000 rw-p 00000000 00:00 0 7f2bc4c6a000-7f2bc4c6c000 r-xp 00000000 fd:00 10758 /lib64/libdl-2.12.so 7f2bc4c6c000-7f2bc4e6c000 ---p 00002000 fd:00 10758 /lib64/libdl-2.12.so 7f2bc4e6c000-7f2bc4e6d000 r--p 00002000 fd:00 10758 /lib64/libdl-2.12.so 7f2bc4e6d000-7f2bc4e6e000 rw-p 00003000 fd:00 10758 /lib64/libdl-2.12.so 7f2bc4e6e000-7f2bc4e75000 r-xp 00000000 fd:00 10772 /lib64/librt-2.12.so 7f2bc4e75000-7f2bc5074000 ---p 00007000 fd:00 10772 /lib64/librt-2.12.so 7f2bc5074000-7f2bc5075000 r--p 00006000 fd:00 10772 /lib64/librt-2.12.so 7f2bc5075000-7f2bc5076000 rw-p 00007000 fd:00 10772 /lib64/librt-2.12.so 7f2bc5076000-7f2bc508d000 r-xp 00000000 fd:00 2913 /lib64/libpthread-2.12.so 7f2bc508d000-7f2bc528d000 ---p 00017000 fd:00 2913 /lib64/libpthread-2.12.so 7f2bc528d000-7f2bc528e000 r--p 00017000 fd:00 2913 /lib64/libpthread-2.12.so 7f2bc528e000-7f2bc528f000 rw-p 00018000 fd:00 2913 /lib64/libpthread-2.12.so 7f2bc528f000-7f2bc5293000 rw-p 00000000 00:00 0 7f2bc5293000-7f2bc5567000 r-xp 00000000 fd:00 26860 /opt/sensu/embedded/lib/libruby.so.2.3.0 7f2bc5567000-7f2bc5766000 ---p 002d4000 fd:00 26860 /opt/sensu/embedded/lib/libruby.so.2.3.0 7f2bc5766000-7f2bc576f000 rw-p 002d3000 fd:00 26860 /opt/sensu/embedded/lib/libruby.so.2.3.0 7f2bc576f000-7f2bc5781000 rw-p 00000000 00:00 0 7f2bc5781000-7f2bc57a1000 r-xp 00000000 fd:00 3327 /lib64/ld-2.12.so 7f2bc5833000-7f2bc5834000 rw-p 00000000 00:00 0 7f2bc5834000-7f2bc583b000 r--s 00000000 fd:00 3252 /usr/lib64/gconv/gconv-modules.cache 7f2bc583b000-7f2bc586d000 r--s 00000000 fd:00 279276 /opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/ffi-1.9.14/lib/ffi_c.so 7f2bc586d000-7f2bc5890000 r--s 00000000 fd:00 2913 /lib64/libpthread-2.12.so 7f2bc5890000-7f2bc5997000 rw-p 00000000 00:00 0 7f2bc5998000-7f2bc599a000 r--s 00000000 fd:00 26685 /opt/sensu/embedded/bin/ruby 7f2bc599a000-7f2bc599b000 r-xp 00000000 00:00 0 7f2bc599b000-7f2bc599c000 ---p 00000000 00:00 0 7f2bc599c000-7f2bc59a0000 rw-p 00000000 00:00 0 7f2bc59a0000-7f2bc59a2000 r--p 0001f000 fd:00 3327 /lib64/ld-2.12.so 7f2bc59a2000-7f2bc59a3000 rw-p 00021000 fd:00 3327 /lib64/ld-2.12.so 7f2bc59a3000-7f2bc59a4000 rw-p 00000000 00:00 0 7ffe334ad000-7ffe33eac000 rw-p 00000000 00:00 0 [stack] 7ffe33ff5000-7ffe33ff6000 r-xp 00000000 00:00 0 [vdso]

ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]

[NOTE] You may have encountered a bug in the Ruby interpreter or extension libraries. Bug reports are welcome. For details: http://www.ruby-lang.org/bugreport.html

ajayrockrock commented 7 years ago

I have the exact same issue after upgrading to RHEL 6.9. Sorry I have nothing to add other than a "me too".

bootswithdefer commented 7 years ago

The underlying cause of the crash has been fixed in RHEL. https://bugzilla.redhat.com/show_bug.cgi?id=1437618 https://access.redhat.com/errata/RHBA-2017:0894

majormoses commented 7 years ago

closing per: https://github.com/sensu-plugins/sensu-plugins-disk-checks/issues/65#issuecomment-299511369 please respond back if you feel this needs to be re-opened.

exeral commented 6 years ago

Hi

Currently the check raise a Critical status if the mounts bug is encountered

What do you think of catching the Filesystem.mounts exception ? so it raise an unknown instead

fs_mounts original:

  def fs_mounts
    Filesystem.mounts.each do |line|
      begin
        next if config[:fstype] && !config[:fstype].include?(line.mount_type)
        next if config[:ignoretype] && config[:ignoretype].include?(line.mount_type)
        next if config[:ignoremnt] && config[:ignoremnt].include?(line.mount_point)
        next if config[:ignorepathre] && config[:ignorepathre].match(line.mount_point)
        next if config[:ignoreopt] && config[:ignoreopt].include?(line.options)
        next if config[:includemnt] && !config[:includemnt].include?(line.mount_point)
      rescue
        unknown 'An error occured getting the mount info'
      end
      check_mount(line)
    end
  end

becomes:

  def fs_mounts
    begin
        Filesystem.mounts.each do |line|
          next if config[:fstype] && !config[:fstype].include?(line.mount_type)
          next if config[:ignoretype] && config[:ignoretype].include?(line.mount_type)
          next if config[:ignoremnt] && config[:ignoremnt].include?(line.mount_point)
          next if config[:ignorepathre] && config[:ignorepathre].match(line.mount_point)
          next if config[:ignoreopt] && config[:ignoreopt].include?(line.options)
          next if config[:includemnt] && !config[:includemnt].include?(line.mount_point)
          check_mount(line)
        end
    rescue
        unknown 'An error occured getting the mount info'
    end
  end

maybe the check_mount should be moved out of the begin/rescue. I don't really know

majormoses commented 6 years ago

Sorry I did not see the response back, must have been lost in all the noise. Yes we totally should be rescuing that and I am releasing a hotfix right now. Thanks for the submission!

majormoses commented 6 years ago

released: https://rubygems.org/gems/sensu-plugins-disk-checks/versions/2.5.1