Segmentation fault Icinga 2.8

arcadesolutions commented 6 years ago

Current Behavior

It is possible to generate a segmentation fault an crash icinga2 with simple GET requests to port 5665 on Centos7

Steps to Reproduce (for bugs)

As for now, i was just able to reproduce it for Icinga 2.8.0, earlier versions doesn't seem to be affected, tested on different version (2.3 - 2.7) all running Centos7

Just send a bunch of GET requests to an Icinga2 Client:

ab -n 2000 -c 10 -f all  https://127.0.0.1:5665/

The crash happens between 2000-3000 requests with the following log entry in the message log:

Dec 18 10:41:59 host-1 kernel: icinga2[30002]: segfault at 10 ip 00007fcd80f162b4 sp 00007fcd66e929a8 error 4 in libstdc++.so.6.0.19[7fcd80ea1000+e9000]
Dec 18 10:41:59 host-1 systemd: icinga2.service: main process exited, code=killed, status=11/SEGV
Dec 18 10:41:59 host-1 systemd: Unit icinga2.service entered failed state.
Dec 18 10:41:59 host-1 systemd: icinga2.service failed.

A Coredump is generated, this are the last output of a backtrace wie gdb:

#0  std::local_Rb_tree_rotate_left (__x=__x@entry=0x10cad00, __root=@0x7fcd5c000df0: 0x7fcd40017cb0) at ../../../../../libstdc++-v3/src/c++98/tree.cc:138
#1  0x00007fcd80f16591 in std::_Rb_tree_insert_and_rebalance (__insert_left=__insert_left@entry=true, __x=0x7fcd78001a00, __p=__p@entry=0x7fcd78001a00, __header=...) at ../../../../../libstdc++-v3/src/c++98/tree.cc:278
#2  0x00007fcd8278cbfc in _M_insert_ (__v=..., __p=0x7fcd78001a00, __x=0x0, this=0x7fcd5c000de0) at /usr/include/c++/4.8.2/bits/stl_tree.h:1025
#3  _M_insert_unique (__v=..., this=0x7fcd5c000de0) at /usr/include/c++/4.8.2/bits/stl_tree.h:1382
#4  insert (this=0x7fcd5c000de0, __x=...) at /usr/include/c++/4.8.2/bits/stl_set.h:463
#5  icinga::ApiListener::AddHttpClient (this=this@entry=0x7fcd5c000c20, aclient=...) at /usr/src/debug/icinga2-2.8.0/lib/remote/apilistener.cpp:1376
#6  0x00007fcd827af4bc in icinga::ApiListener::NewClientHandlerInternal (this=0x7fcd5c000c20, client=..., hostname=..., role=<optimized out>) at /usr/src/debug/icinga2-2.8.0/lib/remote/apilistener.cpp:550
#7  0x00007fcd827b0023 in icinga::ApiListener::NewClientHandler (this=<optimized out>, client=..., hostname=..., role=<optimized out>) at /usr/src/debug/icinga2-2.8.0/lib/remote/apilistener.cpp:409
#8  0x00007fcd83aa827a in boost::(anonymous namespace)::thread_proxy (param=<optimized out>) at libs/thread/src/pthread/thread.cpp:165
#9  0x00007fcd80774dc5 in start_thread (arg=0x7fcd66e94700) at pthread_create.c:308
#10 0x00007fcd804a373d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113

Let me know if you need some more infos.

Context

We had a recent security audit at our company where the icinga2 process crashed during some penetration testing.

Your Environment

Version used (icinga2 --version):

icinga2 - The Icinga 2 network monitoring daemon (version: r2.8.0-1)

Copyright (c) 2012-2017 Icinga Development Team (https://www.icinga.com/)
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl2.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Application information:
  Installation root: /usr
  Sysconf directory: /etc
  Run directory: /run
  Local state directory: /var
  Package data directory: /usr/share/icinga2
  State path: /var/lib/icinga2/icinga2.state
  Modified attributes path: /var/lib/icinga2/modified-attributes.conf
  Objects path: /var/cache/icinga2/icinga2.debug
  Vars path: /var/cache/icinga2/icinga2.vars
  PID path: /run/icinga2/icinga2.pid

System information:
  Platform: CentOS Linux
  Platform version: 7 (Core)
  Kernel: Linux
  Kernel version: 3.10.0-514.el7.x86_64
  Architecture: x86_64

Build information:
  Compiler: GNU 4.8.5
  Build host: unknown

Operating System and version:

CentOS Linux release 7.3.1611 (Core)

Enabled features (icinga2 feature list):

Disabled features: command compatlog debuglog elasticsearch gelf graphite influxdb livestatus notification opentsdb perfdata statusdata syslog
Enabled features: api checker mainlog

gunnarbeutner commented 6 years ago

This is a duplicate of #5807.

arcadesolutions commented 6 years ago

Ah well, should have searched for closed issues too, thanks

Icinga / icinga2