Closed icinga-migration closed 9 years ago
Updated by mfriedrich on 2014-04-29 16:14:51 +00:00
Updated by mfriedrich on 2014-04-29 16:16:37 +00:00
Updated by mfriedrich on 2014-05-09 08:29:41 +00:00
Updated by gbeutner on 2014-05-16 09:45:39 +00:00
Updated by mfriedrich on 2014-05-19 14:08:08 +00:00
Updated by mfriedrich on 2014-05-19 17:47:31 +00:00
Remove
warning/perfdata: GraphiteWriter already connected on socket on host '127.0.0.1' port '2003'.
Updated by mfriedrich on 2014-05-22 12:12:05 +00:00
Updated by mfriedrich on 2014-05-22 14:13:56 +00:00
a) is open for discussion b) is done for the most common connection errors (api listener, db ido*). requires more love extracting error messages from boost::exception c) is implemented, documented and tested
I've moved all cluster log replay, log rotate, external commands and other change indications to 'notice' severity.
[2014-05-22 16:11:41 +0200] critical/remote: Cannot connect to host 'localhost' on port '8000'
[2014-05-22 16:12:00 +0200] critical/db_ido_mysql: Exception during database operation: 'Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)'
[2014-05-22 16:11:41 +0200] notice/remote: Replaying log: icinga2a/lib/icinga2/api/log/1400767899
[2014-05-22 16:11:41 +0200] notice/remote: Replaying log: icinga2a/lib/icinga2/api/log/1400767901
[2014-05-22 16:11:42 +0200] notice/remote: Replayed 1 messages.
[2014-05-22 16:11:42 +0200] notice/remote: Replaying log: icinga2a/lib/icinga2/api/log/1400767901
[2014-05-22 16:11:42 +0200] notice/remote: Replaying log: icinga2a/lib/icinga2/api/log/1400767902
[2014-05-22 16:11:42 +0200] notice/remote: Replayed 0 messages.
[2014-05-22 16:11:45 +0200] notice/base: Pool #1: Pending tasks: 0; Average latency: 1ms; Threads: 4; Pool utilization: 2.68703%
[2014-05-22 16:11:47 +0200] notice/remote: Setting log position for identity 'icinga2c': 2014/05/22 16:11:46
[2014-05-22 16:11:53 +0200] notice/remote: Setting log position for identity 'icinga2c': 2014/05/22 16:11:53
[2014-05-22 16:12:00 +0200] notice/remote: Setting log position for identity 'icinga2c': 2014/05/22 16:11:56
[2014-05-22 16:12:00 +0200] notice/base: Pool #1: Pending tasks: 0; Average latency: 24ms; Threads: 4; Pool utilization: 5.42124%
[2014-05-22 16:12:06 +0200] notice/remote: Setting log position for identity 'icinga2c': 2014/05/22 16:12:01
[2014-05-22 16:12:12 +0200] notice/remote: Setting log position for identity 'icinga2c': 2014/05/22 16:12:11
[2014-05-22 16:12:15 +0200] notice/base: Pool #1: Pending tasks: 0; Average latency: 0ms; Threads: 4; Pool utilization: 3.1198%
[2014-05-22 16:12:18 +0200] notice/remote: Setting log position for identity 'icinga2c': 2014/05/22 16:12:18
[2014-05-22 16:12:24 +0200] notice/remote: Setting log position for identity 'icinga2c': 2014/05/22 16:12:24
[2014-05-22 16:12:30 +0200] notice/remote: Setting log position for identity 'icinga2c': 2014/05/22 16:12:26
[2014-05-22 16:12:30 +0200] notice/base: Pool #1: Pending tasks: 0; Average latency: 32ms; Threads: 4; Pool utilization: 6.91768%
Updated by mfriedrich on 2014-05-26 11:37:24 +00:00
TODO
Updated by mfriedrich on 2014-05-26 12:41:16 +00:00
[2014-05-26 14:31:15 +0200] information/config: Compiling config file: /etc/icinga2/conf.d/hosts/localhost/swap.conf
[2014-05-26 14:31:15 +0200] information/config: Compiling config file: /etc/icinga2/conf.d/hosts/localhost/users.conf
[2014-05-26 14:31:15 +0200] information/config: Validating config items (step 1)...
[2014-05-26 14:31:15 +0200] information/config: Committing config items
[2014-05-26 14:31:15 +0200] information/config: Triggering OnConfigLoaded signal for config items
Caught unhandled exception.
Current time: 2014-05-26 14:31:15 +0200
***
* Application version: 0.0.11+icingasnap201405261042.4775bd8~wheezy
* Installation root: /usr
* Sysconf directory: /etc
* Local state directory: /var
* Package data directory: /usr/share/icinga2
* State path: /var/lib/icinga2/icinga2.state
* PID path: /var/run/icinga2/icinga2.pid
* Application type: icinga/IcingaApplication
***
../../../lib/base/tlsutility.cpp(196): Throw in function boost::shared_ptr icinga::GetX509Certificate(const icinga::String&)
Dynamic exception type: boost::exception_detail::clone_impl
std::exception::what: std::exception
[icinga::StackTrace*] =
(0) libbase.so: void boost::throw_exception(icinga::openssl_error const&) (+0x99) [0x7f4cd4ee1809] (??:0)
(1) libbase.so: void boost::exception_detail::throw_exception_(icinga::openssl_error const&, char const*, char const*, int) (+0x3c) [0x7f4cd4ee18ac] (??:0)
(2) libbase.so: icinga::GetX509Certificate(icinga::String const&) (+0x271) [0x7f4cd4ee2781] (??:0)
(3) libremote.so: icinga::ApiListener::OnConfigLoaded() (+0x40) [0x7f4cd08dd990] (??:0)
(4) libbase.so: icinga::WorkQueue::WorkerThreadProc() (+0x354) [0x7f4cd4ef68f4] (??:0)
(5) libboost_thread.so.1.49.0: (+0x10629) [0x7f4cd556f629] (??:0)
(6) libpthread.so.0: (+0x6b50) [0x7f4cd5155b50] (??:0)
(7) libc.so.6: clone (+0x6d) [0x7f4cd31f70ed] (??:0)
[icinga::ContextTrace*] =
[icinga::errinfo_openssl_error_*] = 33558530, "error:02001002:lib(2):func(1):reason(2)"
[boost::errinfo_file_name_*] = /etc/icinga2/pki/localhost.crt
[boost::errinfo_api_function_*] = PEM_read_bio_X509_AUX
***
* This would indicate a runtime problem or configuration error. If you believe this is a bug in Icinga 2
* please submit a bug report at https://dev.icinga.org/ and include this stack trace as well as any other
* information that might be useful in order to reproduce this problem.
***
Aborted
[FAIL] checking Icinga2 configuration ... failed!
Updated by mfriedrich on 2014-05-26 19:31:04 +00:00
[2014-05-26 21:30:26 +0200] information/config: Evaluating 'object' rules (step 1)...
[2014-05-26 21:30:26 +0200] information/config: Evaluating 'apply' rules...
Caught unhandled exception.
Current time: 2014-05-26 21:30:26 +0200
***
* Application version: v0.0.11-98-g08aa22a
* Installation root: /usr
* Sysconf directory: /etc
* Local state directory: /var
* Package data directory: /usr/share/icinga2
* State path: /var/lib/icinga2/icinga2.state
* PID path: /var/run/icinga2/icinga2.pid
* Application type: icinga/IcingaApplication
***
/home/michi/coding/icinga/icinga2/lib/config/aexpression.cpp(64): Throw in function icinga::Value icinga::AExpression::Evaluate(const Dictionary::Ptr &) const
Dynamic exception type: boost::exception_detail::clone_impl
std::exception::what: Error while evaluating expression: Tried to access invalid property 'group'
Config location: in /etc/icinga2/conf.d/tests/apply-exceptions.conf: 49:38-49:47
/etc/icinga2/conf.d/tests/apply-exceptions.conf(47): vars.ssh_port = "$host.vars.custom_ssh_port$"
/etc/icinga2/conf.d/tests/apply-exceptions.conf(48):
/etc/icinga2/conf.d/tests/apply-exceptions.conf(49): assign where "my-linux-servers" in host.group
^^^^^^^^^^
/etc/icinga2/conf.d/tests/apply-exceptions.conf(50): }
/etc/icinga2/conf.d/tests/apply-exceptions.conf(51):
[boost::errinfo_nested_exception_*] =
/home/michi/coding/icinga/icinga2/lib/config/aexpression.cpp(449): Throw in function static icinga::Value icinga::AExpression::OpIndexer(const icinga::AExpression *, const Dictionary::Ptr &)
Dynamic exception type: boost::exception_detail::clone_impl
std::exception::what: Tried to access invalid property 'group'
***
* This would indicate a runtime problem or configuration error. If you believe this is a bug in Icinga 2
* please submit a bug report at https://dev.icinga.org/ and include this stack trace as well as any other
* information that might be useful in order to reproduce this problem.
***
Aborted
Not restarting Icinga 2 due to configuration errors.
Updated by mfriedrich on 2014-05-26 19:32:24 +00:00
the apply error above is a rule specific error.
this one generates a duplicated service object.
[2014-05-26 21:31:23 +0200] information/config: Evaluating 'object' rules (step 1)...
[2014-05-26 21:31:23 +0200] information/config: Evaluating 'apply' rules...
Caught unhandled exception.
Current time: 2014-05-26 21:31:23 +0200
***
* Application version: v0.0.11-98-g08aa22a
* Installation root: /usr
* Sysconf directory: /etc
* Local state directory: /var
* Package data directory: /usr/share/icinga2
* State path: /var/lib/icinga2/icinga2.state
* PID path: /var/run/icinga2/icinga2.pid
* Application type: icinga/IcingaApplication
***
/home/michi/coding/icinga/icinga2/lib/base/dynamictype.cpp(102): Throw in function void icinga::DynamicType::RegisterObject(const DynamicObject::Ptr &)
Dynamic exception type: boost::exception_detail::clone_impl >
std::exception::what: RegisterObject() found existing object with the same name: localhost!ssh
[icinga::StackTrace*] =
(0) libbase.so: void boost::throw_exception >(boost::exception_detail::error_info_injector const&) (+0x6e) [0x7fb2df95f09e] (throw_exception.hpp:67)
(1) libbase.so: void boost::exception_detail::throw_exception_(std::runtime_error const&, char const*, char const*, int) (+0xb9) [0x7fb2df951829] (throw_exception.hpp:84)
(2) libbase.so: icinga::DynamicType::RegisterObject(boost::shared_ptr const&) (+0x189) [0x7fb2df99a679] (dynamictype.cpp:102)
(3) libbase.so: icinga::DynamicObject::Register() (+0x6e) [0x7fb2df98b01e] (dynamicobject.cpp:104)
(4) libconfig.so: icinga::ConfigItem::Commit() (+0x59a) [0x7fb2df46a6da] (configitem.cpp:188)
(5) libicinga.so: icinga::Service::EvaluateApplyRuleOne(boost::shared_ptr const&, icinga::ApplyRule const&) (+0xd5f) [0x7fb2daa7d9ff] (service-apply.cpp:87)
(6) libicinga.so: icinga::Service::EvaluateApplyRule(icinga::ApplyRule const&) (+0x4b5) [0x7fb2daa7e0c5] (service-apply.cpp:100)
(7) libicinga.so: void boost::_bi::list1 >::operator()(boost::_bi::type, void (*&)(icinga::ApplyRule const&), boost::_bi::list0&, int) (+0x55) [0x7fb2da9ce3b5] (bind.hpp:254)
(8) libicinga.so: boost::_bi::bind_t > >::operator()() (+0x3f) [0x7fb2da9ce34f] (bind_template.hpp:20)
(9) libicinga.so: boost::detail::function::void_function_obj_invoker0 > >, void>::invoke(boost::detail::function::function_buffer&) (+0x1d) [0x7fb2da9ce13d] (function_template.hpp:154)
(10) libbase.so: boost::function0::operator()() const (+0x7b) [0x7fb2df974cfb] (function_template.hpp:766)
(11) libbase.so: icinga::WorkQueue::WorkerThreadProc() (+0x5a0) [0x7fb2dfa394b0] (workqueue.cpp:157)
(12) libbase.so: boost::_mfi::mf0::operator()(icinga::WorkQueue*) const (+0x72) [0x7fb2dfa3bec2] (mem_fn_template.hpp:49)
(13) libbase.so: void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) (+0x55) [0x7fb2dfa3be25] (bind.hpp:254)
(14) libbase.so: boost::_bi::bind_t, boost::_bi::list1 > >::operator()() (+0x3f) [0x7fb2dfa3bdbf] (bind_template.hpp:20)
(15) libbase.so: boost::detail::thread_data, boost::_bi::list1 > > >::run() (+0x1c) [0x7fb2dfa3bd6c] (thread.hpp:118)
(16) libboost_thread.so.1.54.0: (+0xc29a) [0x7fb2e06eb29a] (??:?)
(17) libpthread.so.0: (+0x8062) [0x7fb2dfd48062] (pthread_create.c:312)
(18) libc.so.6: clone (+0x6d) [0x7fb2dd997bfd] (clone.S:113)
[icinga::ContextTrace*] =
(0) Evaluating 'apply' rule (in /etc/icinga2/conf.d/tests/apply-exceptions.conf: 42:1-42:19)
(1) Evaluating 'apply' rules for host 'localhost'
***
* This would indicate a runtime problem or configuration error. If you believe this is a bug in Icinga 2
* please submit a bug report at https://dev.icinga.org/ and include this stack trace as well as any other
* information that might be useful in order to reproduce this problem.
***
Aborted
Not restarting Icinga 2 due to configuration errors.
Updated by mfriedrich on 2014-05-28 11:36:50 +00:00
Updated by mfriedrich on 2014-05-28 11:37:03 +00:00
Updated by mfriedrich on 2014-05-28 17:45:16 +00:00
[2014-05-28 19:42:39 +0200] information/ExternalCommandListener: Executing external command: [1401298959] ACKNOWLEDGE_SVC_PROBLEM;"FOO";"ping response";2;1;1;system;"Home-Office
[2014-05-28 19:42:39 +0200] critical/ThreadPool: Exception thrown in event handler:
Throw location unknown (consider using BOOST_THROW_EXCEPTION)
Dynamic exception type: boost::exception_detail::clone_impl >
std::exception::what: call to empty boost::function
[icinga::StackTrace*] =
(0) icinga2: void boost::throw_exception(boost::bad_function_call const&) (+0x126) [0x433096] (??:0)
(1) libicinga.so: boost::detail::function::void_function_obj_invoker1, boost::_bi::l
ist2, boost::arg<1> > >, void, icinga::ProcessResult const&>::invoke(boost::detail::function::function_buffer&, icinga::ProcessResult const&) (+0x4a) [0x7f3fc420be5a] (??:0)
(2) libbase.so: boost::detail::function::void_function_obj_invoker0, boost::_bi::list1 > >, void>::invoke(boost::detail::function::function_buffer&) (+0x3f) [0x7f3fc6b1d07f] (??:0)
(3) libbase.so: icinga::ThreadPool::WorkerThread::ThreadProc(icinga::ThreadPool::Queue&) (+0x444) [0x7f3fc6b46434] (??:0)
(4) libboost_thread.so.1.54.0: (+0xd5e3) [0x7f3fc76645e3] (thread.cpp:?)
(5) libpthread.so.0: (+0x77f6) [0x7f3fc4d677f6] (??:?)
(6) libc.so.6: clone (+0x6d) [0x7f3fc526109d] (??:?)
[icinga::ContextTrace*] =
[2014-05-28 19:42:39 +0200] warning/ExternalCommandListener: External command failed: /usr/src/packages/BUILD/icinga2-abuild/lib/icinga/externalcommandprocessor.cpp(629): Throw in function static void icinga::Ex
ternalCommandProcessor::AcknowledgeSvcProblem(double, const std::vector >&)
Dynamic exception type: boost::exception_detail::clone_impl >
std::exception::what: Cannot acknowledge service problem for non-existent service '"ping response"' on host '"FOO"'
[icinga::StackTrace*] =
(0) icinga2: void boost::exception_detail::throw_exception_(std::invalid_argument const&, char const*, char const*, int) (+0x197) [0x432ae7] (??:0)
(1) libicinga.so: icinga::ExternalCommandProcessor::AcknowledgeSvcProblem(double, std::vector > const&) (+0x37f) [0x7f3fc41c284f] (??:0)
(2) libicinga.so: icinga::ExternalCommandProcessor::Execute(double, icinga::String const&, std::vector > const&) (+0x553) [0x7f3fc41bf493] (??:0)
(3) libicinga.so: icinga::ExternalCommandProcessor::Execute(icinga::String const&) (+0x309) [0x7f3fc41c0209] (??:0)
(4) libcompat.so: icinga::ExternalCommandListener::CommandPipeThread(icinga::String const&) (+0x23d) [0x7f3fc32f479d] (??:0)
(5) libboost_thread.so.1.54.0: (+0xd5e3) [0x7f3fc76645e3] (thread.cpp:?)
(6) libpthread.so.0: (+0x77f6) [0x7f3fc4d677f6] (??:?)
(7) libc.so.6: clone (+0x6d) [0x7f3fc526109d] (??:?)
[icinga::ContextTrace*] =
/var/log/icinga2/icinga2.log lines 127500-127549/127549 (END)
For some reason, Icinga Web 1.11 is sending double quoted strings to the command pipe.
Updated by mfriedrich on 2014-06-05 14:35:29 +00:00
Seems we are lazy programmers. Even icinga-app contains exceptions only and no logging, nor a try catch block around Main().
nbmif /etc/icinga2 (master) # icinga2 -c icinga2.conf -u foo -x notice
[2014-06-05 16:33:20 +0200] critical/icinga-app: Invalid user specified: foo
nbmif /etc/icinga2 (master) # icinga2 -c icinga2.conf -u icinga -g bar -x notice
[2014-06-05 16:33:30 +0200] critical/icinga-app: Invalid group specified: bar
Updated by mfriedrich on 2014-06-05 16:26:00 +00:00
[2014-06-05 18:25:35 +0200] information/IdoMysqlConnection: MySQL IDO instance id: 1 (schema version: '1.11.3')
[2014-06-05 18:25:35 +0200] information/IdoPgsqlConnection: pgSQL IDO instance id: 1 (schema version: '1.11.0')
[2014-06-05 18:25:35 +0200] critical/IdoMysqlConnection: Error "Unknown column 'is_reachable' in 'field list'" when executing query "UPDATE icinga_hoststatus SET acknowledgement_type = '0', active_checks_enabled = '1', check_command = 'hostalive', check_source = 'nbmif', check_type = '0', current_check_attempt = '1', current_notification_number = '0', current_state = '2', event_handler = '', event_handler_enabled = '0', execution_time = '4.1595048904418945', flap_detection_enabled = '1', has_been_checked = '1', host_object_id = 1461, instance_id = 1, is_flapping = '0', is_reachable = '0', last_check = FROM_UNIXTIME(1400578246), last_hard_state_change = FROM_UNIXTIME(1400578246), last_state_change = FROM_UNIXTIME(1400578197), latency = '0', long_output = '', max_check_attempts = '3', modified_host_attributes = '0', next_check = FROM_UNIXTIME(1401985743), normal_check_interval = '5', notifications_enabled = '1', output = '', passive_checks_enabled = '1', percent_state_change = '0', perfdata = '', problem_has_been_acknowledged = '0', process_performance_data = '0', retry_check_interval = '1', scheduled_downtime_depth = '0', should_be_scheduled = '1', state_type = '1', status_update_time = FROM_UNIXTIME(1401985535) WHERE host_object_id = 1461"
Context:
(0) Reconnecting to MySQL IDO database 'ido-mysql'
[2014-06-05 18:25:35 +0200] critical/IdoMysqlConnection: Exception during database operation: Verify that your database is operational!
[2014-06-05 18:25:36 +0200] critical/IdoPgsqlConnection: Error "FEHLER: Spalte „is_reachable“ von Relation „icinga_hoststatus“ existiert nicht
LINE 1: ...d = 12063, instance_id = 1, is_flapping = '0', is_reachab...
^
" when executing query "UPDATE icinga_hoststatus SET acknowledgement_type = '0', active_checks_enabled = '1', check_command = 'hostalive', check_source = 'nbmif', check_type = '0', current_check_attempt = '1', current_notification_number = '0', current_state = '2', event_handler = '', event_handler_enabled = '0', execution_time = '4.1595048904418945', flap_detection_enabled = '1', has_been_checked = '1', host_object_id = 12063, instance_id = 1, is_flapping = '0', is_reachable = '0', last_check = TO_TIMESTAMP(1400578246), last_hard_state_change = TO_TIMESTAMP(1400578246), last_state_change = TO_TIMESTAMP(1400578197), latency = '0', long_output = '', max_check_attempts = '3', modified_host_attributes = '0', next_check = TO_TIMESTAMP(1401985743), normal_check_interval = '5', notifications_enabled = '1', output = '', passive_checks_enabled = '1', percent_state_change = '0', perfdata = '', problem_has_been_acknowledged = '0', process_performance_data = '0', retry_check_interval = '1', scheduled_downtime_depth = '0', should_be_scheduled = '1', state_type = '1', status_update_time = TO_TIMESTAMP(1401985536) WHERE host_object_id = 12063"
Context:
(0) Reconnecting to PostgreSQL IDO database 'ido-pgsql'
[2014-06-05 18:25:36 +0200] warning/IdoPgsqlConnection: Exception during database operation: Verify that your database is operational!
Updated by mfriedrich on 2014-06-06 08:52:30 +00:00
Updated by mfriedrich on 2014-06-10 21:27:45 +00:00
Configuration - apply rules which would generate duplicated objects (imho they shouldn't register another object, but ignore the rule)
michi@imagine ~/coding/icinga/icinga2-migration (master) $ ./run
[2014-06-10 23:19:08 +0200] information/icinga-app: Icinga application loader (version: v2.0.0-beta2-3-gaed88c6)
[2014-06-10 23:19:08 +0200] information/icinga-app: Loading application type: icinga/IcingaApplication
[2014-06-10 23:19:08 +0200] information/Utility: Loading library 'libicinga.so'
[2014-06-10 23:19:08 +0200] information/ConfigCompiler: Adding include search dir: /usr/share/icinga2/include
[2014-06-10 23:19:08 +0200] information/ConfigCompiler: Compiling config file: /tmp/icinga2.conf
[2014-06-10 23:19:08 +0200] information/ConfigCompiler: Compiling config file: /usr/share/icinga2/include/itl
[2014-06-10 23:19:08 +0200] information/ConfigCompiler: Compiling config file: /usr/share/icinga2/include/command.conf
[2014-06-10 23:19:08 +0200] information/Utility: Loading library 'libmethods.so'
[2014-06-10 23:19:08 +0200] information/ConfigCompiler: Compiling config file: /usr/share/icinga2/include/command-icinga.conf
[2014-06-10 23:19:08 +0200] information/ConfigCompiler: Compiling config file: /usr/share/icinga2/include/timeperiod.conf
[2014-06-10 23:19:08 +0200] information/ConfigCompiler: Compiling config file: /usr/share/icinga2/include/plugins
[2014-06-10 23:19:08 +0200] information/ConfigCompiler: Compiling config file: /usr/share/icinga2/include/command-plugins.conf
[2014-06-10 23:19:08 +0200] information/ConfigCompiler: Compiling config file: /tmp/migrate.conf
[2014-06-10 23:19:08 +0200] information/ConfigItem: Validating config items (step 1)...
[2014-06-10 23:19:08 +0200] information/ConfigItem: Committing config items
[2014-06-10 23:19:08 +0200] information/ConfigItem: Triggering OnConfigLoaded signal for config items
[2014-06-10 23:19:08 +0200] information/ConfigItem: Evaluating 'object' rules (step 1)...
[2014-06-10 23:19:08 +0200] information/ConfigItem: Evaluating 'apply' rules...
Caught unhandled exception.
Current time: 2014-06-10 23:19:08 +0200
***
* Application version: v2.0.0-beta2-3-gaed88c6
* Installation root: /usr
* Sysconf directory: /etc
* Local state directory: /var
* Package data directory: /usr/share/icinga2
* State path: /var/lib/icinga2/icinga2.state
* PID path: /var/run/icinga2/icinga2.pid
* Application type: icinga/IcingaApplication
***
Caught unhandled exception.
Current time: 2014-06-10 23:19:08 +0200
***
* Application version: v2.0.0-beta2-3-gaed88c6
* Installation root: /usr
* Sysconf directory: /etc
* Local state directory: /var
* Package data directory: /usr/share/icinga2
* State path: /var/lib/icinga2/icinga2.state
* PID path: /var/run/icinga2/icinga2.pid
* Application type: icinga/IcingaApplication
***
Caught unhandled exception.
Current time: 2014-06-10 23:19:08 +0200
***
* Application version: v2.0.0-beta2-3-gaed88c6
* Installation root: /usr
* Sysconf directory: /etc
* Local state directory: /var
* Package data directory: /usr/share/icinga2
* State path: /var/lib/icinga2/icinga2.state
* PID path: /var/run/icinga2/icinga2.pid
* Application type: icinga/IcingaApplication
***
Caught unhandled exception.
Current time: 2014-06-10 23:19:08 +0200
***
* Application version: v2.0.0-beta2-3-gaed88c6
* Installation root: /usr
* Sysconf directory: /etc
* Local state directory: /var
* Package data directory: /usr/share/icinga2
* State path: /var/lib/icinga2/icinga2.state
* PID path: /var/run/icinga2/icinga2.pid
* Application type: icinga/IcingaApplication
***
Caught unhandled exception.
Current time: 2014-06-10 23:19:08 +0200
***
* Application version: v2.0.0-beta2-3-gaed88c6
* Installation root: /usr
* Sysconf directory: /etc
* Local state directory: /var
* Package data directory: /usr/share/icinga2
* State path: /var/lib/icinga2/icinga2.state
* PID path: /var/run/icinga2/icinga2.pid
* Application type: icinga/IcingaApplication
***
Caught unhandled exception.
Current time: 2014-06-10 23:19:08 +0200
***
* Application version: v2.0.0-beta2-3-gaed88c6
* Installation root: /usr
* Sysconf directory: /etc
* Local state directory: /var
* Package data directory: /usr/share/icinga2
* State path: /var/lib/icinga2/icinga2.state
* PID path: /var/run/icinga2/icinga2.pid
* Application type: icinga/IcingaApplication
***
Caught unhandled exception.
Current time: 2014-06-10 23:19:08 +0200
***
* Application version: v2.0.0-beta2-3-gaed88c6
* Installation root: /usr
* Sysconf directory: /etc
* Local state directory: /var
* Package data directory: /usr/share/icinga2
* State path: /var/lib/icinga2/icinga2.state
* PID path: /var/run/icinga2/icinga2.pid
* Application type: icinga/IcingaApplication
***
/home/michi/coding/icinga/icinga2/lib/base/dynamictype.cpp(102): Throw in function void icinga::DynamicType::RegisterObject(const Ptr&)
Dynamic exception type: N5boost16exception_detail10clone_implINS0_19error_info_injectorISt13runtime_errorEEEE
std::exception::what: RegisterObject() found existing object with the same name: 2743localhost3!2743servicewithexcludes
[PN6icinga10StackTraceE] =
(0) libbase.so: void boost::throw_exception >(boost::exception_detail::error_info_injector const&) (+0x72) [0x7f17a08166fa] (throw_exception.hpp:70 (discriminator 2))
(1) libbase.so: void boost::exception_detail::throw_exception_(std::runtime_error const&, char const*, char const*, int) (+0xb6) [0x7f17a08133c1] (throw_exception.hpp:87 (discriminator 1))
(2) libbase.so: icinga::DynamicType::RegisterObject(boost::shared_ptr const&) (+0x125) [0x7f17a0841dfb] (dynamictype.cpp:105)
(3) libbase.so: icinga::DynamicObject::Register() (+0x61) [0x7f17a0836483] (dynamicobject.cpp:104 (discriminator 1))
(4) libconfig.so: icinga::ConfigItem::Commit() (+0x337) [0x7f17a03019c3] (configitem.cpp:190)
(5) libicinga.so: icinga::Service::EvaluateApplyRuleOne(boost::shared_ptr const&, icinga::ApplyRule const&) (+0x7c1) [0x7f179c0eb801] (service-apply.cpp:88)
(6) libicinga.so: icinga::Service::EvaluateApplyRule(icinga::ApplyRule const&) (+0x28e) [0x7f179c0ebe22] (service-apply.cpp:100 (discriminator 1))
(7) libicinga.so: void boost::_bi::list1 >::operator()(boost::_bi::type, void (*&)(icinga::ApplyRule const&), boost::_bi::list0&, int) (+0x44) [0x7f179c0655c6] (bind.hpp:254)
(8) libicinga.so: boost::_bi::bind_t > >::operator()() (+0x39) [0x7f179c065165] (bind_template.hpp:21)
(9) libicinga.so: boost::detail::function::void_function_obj_invoker0 > >, void>::invoke(boost::detail::function::function_buffer&) (+0x20) [0x7f179c064e95] (function_template.hpp:154)
(10) libbase.so: boost::function0::operator()() const (+0x52) [0x7f17a08288a6] (function_template.hpp:767)
(11) libbase.so: icinga::WorkQueue::WorkerThreadProc() (+0x35c) [0x7f17a08c33fc] (workqueue.cpp:155)
(12) libbase.so: boost::_mfi::mf0::operator()(icinga::WorkQueue*) const (+0x65) [0x7f17a08cfb9f] (mem_fn_template.hpp:50 (discriminator 3))
(13) libbase.so: void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) (+0x4a) [0x7f17a08d1986] (bind.hpp:254)
(14) libbase.so: boost::_bi::bind_t, boost::_bi::list1 > >::operator()() (+0x39) [0x7f17a08d0d37] (bind_template.hpp:21)
(15) libbase.so: boost::detail::thread_data, boost::_bi::list1 > > >::run() (+0x1e) [0x7f17a08cffac] (thread.hpp:118)
(16) libboost_thread.so.1.55.0: (+0xc74a) [0x7f17a156c74a] (??:?)
(17) libpthread.so.0: (+0x8062) [0x7f17a0bc0062] (pthread_create.c:312)
(18) libc.so.6: clone (+0x6d) [0x7f179e7fdc1d] (clone.S:113)
[PN6icinga12ContextTraceE] =
(0) Evaluating 'apply' rule (in /tmp/migrate.conf: 924:1-924:39)
(1) Evaluating 'apply' rules for host '2743localhost3'
***
* This would indicate a runtime problem or configuration error. If you believe this is a bug in Icinga 2
* please submit a bug report at https://dev.icinga.org/ and include this stack trace as well as any other
* information that might be useful in order to reproduce this problem.
***
./run: Zeile 19: 31234 Abgebrochen /usr/sbin/icinga2 -c $ICINGA2CONF -C
Updated by mfriedrich on 2014-06-11 15:22:42 +00:00
Ongoing task.
Updated by mfriedrich on 2014-06-11 15:22:49 +00:00
Updated by mfriedrich on 2014-07-10 08:21:52 +00:00
Updated by mfriedrich on 2014-07-16 07:52:28 +00:00
For non-existing scriptvariables:
apply Dependency "global-dns-failure-service" to Service {
//import "..."
parent_host_name = "global-dns-server"
parent_service_name = "global-dns-health"
states = [ Ok ]
disable_checks = true
disable_notifications = true
assign where match("*", service.name)
}
nbmif /etc/icinga2/conf.d (master) # /etc/init.d/icinga2 checkconfig
Checking configuration: [2014-07-16 09:50:51 +0200] information/icinga-app: Icinga application loader (version: v2.0.0-47-gf9075a0)
[2014-07-16 09:50:51 +0200] information/icinga-app: Loading application type: icinga/IcingaApplication
[2014-07-16 09:50:51 +0200] information/Utility: Loading library 'libicinga.so'
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Adding include search dir: /usr/share/icinga2/include
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /etc/icinga2/icinga2.conf
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /etc/icinga2/constants.conf
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /usr/share/icinga2/include/itl
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /usr/share/icinga2/include/command.conf
[2014-07-16 09:50:51 +0200] information/Utility: Loading library 'libmethods.so'
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /usr/share/icinga2/include/command-icinga.conf
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /usr/share/icinga2/include/timeperiod.conf
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /usr/share/icinga2/include/plugins
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /usr/share/icinga2/include/command-plugins.conf
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /etc/icinga2/features-enabled/checker.conf
[2014-07-16 09:50:51 +0200] information/Utility: Loading library 'libchecker.so'
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /etc/icinga2/features-enabled/command.conf
[2014-07-16 09:50:51 +0200] information/Utility: Loading library 'libcompat.so'
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /etc/icinga2/features-enabled/livestatus.conf
[2014-07-16 09:50:51 +0200] information/Utility: Loading library 'liblivestatus.so'
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /etc/icinga2/features-enabled/mainlog.conf
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /etc/icinga2/features-enabled/notification.conf
[2014-07-16 09:50:51 +0200] information/Utility: Loading library 'libnotification.so'
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /etc/icinga2/features-enabled/statusdata.conf
[2014-07-16 09:50:51 +0200] information/Utility: Loading library 'libcompat.so'
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /etc/icinga2/conf.d/address-override.conf
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /etc/icinga2/conf.d/commands.conf
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /etc/icinga2/conf.d/dns-dep.conf
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /etc/icinga2/conf.d/downtimes.conf
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /etc/icinga2/conf.d/groups.conf
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /etc/icinga2/conf.d/notifications.conf
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /etc/icinga2/conf.d/services.conf
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /etc/icinga2/conf.d/templates.conf
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /etc/icinga2/conf.d/timeperiods.conf
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /etc/icinga2/conf.d/users.conf
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /etc/icinga2/conf.d/hosts/localhost.conf
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /etc/icinga2/conf.d/hosts/localhost/disk.conf
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /etc/icinga2/conf.d/hosts/localhost/http.conf
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /etc/icinga2/conf.d/hosts/localhost/icinga.conf
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /etc/icinga2/conf.d/hosts/localhost/load.conf
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /etc/icinga2/conf.d/hosts/localhost/procs.conf
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /etc/icinga2/conf.d/hosts/localhost/ssh.conf
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /etc/icinga2/conf.d/hosts/localhost/swap.conf
[2014-07-16 09:50:51 +0200] information/ConfigCompiler: Compiling config file: /etc/icinga2/conf.d/hosts/localhost/users.conf
[2014-07-16 09:50:51 +0200] information/ConfigItem: Validating config items (step 1)...
[2014-07-16 09:50:51 +0200] information/ConfigItem: Committing config items
[2014-07-16 09:50:51 +0200] information/ConfigItem: Triggering OnConfigLoaded signal for config items
[2014-07-16 09:50:51 +0200] information/ConfigItem: Evaluating 'object' rules (step 1)...
[2014-07-16 09:50:51 +0200] information/ConfigItem: Evaluating 'apply' rules...
Caught unhandled exception.
Current time: 2014-07-16 09:50:51 +0200
***
* Application version: v2.0.0-47-gf9075a0
* Installation root: /usr
* Sysconf directory: /etc
* Local state directory: /var
* Package data directory: /usr/share/icinga2
* State path: /var/lib/icinga2/icinga2.state
* PID path: /var/run/icinga2/icinga2.pid
* Application type: icinga/IcingaApplication
***
/home/michi/coding/icinga/icinga2/lib/config/aexpression.cpp(64): Throw in function icinga::Value icinga::AExpression::Evaluate(const Dictionary::Ptr &) const
Dynamic exception type: N5boost16exception_detail10clone_implIN6icinga11ConfigErrorEEE
std::exception::what: Error while evaluating expression: Tried to access undefined script variable 'Ok'
Config location: in /etc/icinga2/conf.d/dns-dep.conf: 32:14-32:15
/etc/icinga2/conf.d/dns-dep.conf(30): parent_service_name = "global-dns-health"
/etc/icinga2/conf.d/dns-dep.conf(31):
/etc/icinga2/conf.d/dns-dep.conf(32): states = [ Ok ]
^^
/etc/icinga2/conf.d/dns-dep.conf(33): disable_checks = true
/etc/icinga2/conf.d/dns-dep.conf(34): disable_notifications = true
[PN5boost25errinfo_nested_exception_E] =
/home/michi/coding/icinga/icinga2/lib/base/scriptvariable.cpp(59): Throw in function static icinga::Value icinga::ScriptVariable::Get(const icinga::String &)
Dynamic exception type: N5boost16exception_detail10clone_implINS0_19error_info_injectorISt16invalid_argumentEEEE
std::exception::what: Tried to access undefined script variable 'Ok'
[PN6icinga10StackTraceE] =
(0) libbase.so: void boost::throw_exception >(boost::exception_detail::error_info_injector const&) (+0x6c) [0x7f13dba9a2bc]
(1) libbase.so: void boost::exception_detail::throw_exception_(std::invalid_argument const&, char const*, char const*, int) (+0xb9) [0x7f13dba9a229]
(2) libbase.so: icinga::ScriptVariable::Get(icinga::String const&) (+0xbd) [0x7f13dbaee71d]
(3) libconfig.so: icinga::AExpression::OpVariable(icinga::AExpression const*, boost::shared_ptr const&) (+0x24a) [0x7f13db4b551a]
(4) libconfig.so: icinga::AExpression::Evaluate(boost::shared_ptr const&) const (+0x2e) [0x7f13db4b3f5e]
(5) libconfig.so: icinga::AExpression::OpArray(icinga::AExpression const*, boost::shared_ptr const&) (+0x15e) [0x7f13db4b744e]
(6) libconfig.so: icinga::AExpression::Evaluate(boost::shared_ptr const&) const (+0x2e) [0x7f13db4b3f5e]
(7) libconfig.so: icinga::AExpression::EvaluateOperand2(boost::shared_ptr const&) const (+0x59) [0x7f13db4b5239]
(8) libconfig.so: icinga::AExpression::OpSet(icinga::AExpression const*, boost::shared_ptr const&) (+0x40) [0x7f13db4b75f0]
(9) libconfig.so: icinga::AExpression::Evaluate(boost::shared_ptr const&) const (+0x2e) [0x7f13db4b3f5e]
(10) libconfig.so: icinga::AExpression::OpDict(icinga::AExpression const*, boost::shared_ptr const&) (+0x248) [0x7f13db4b4498]
(11) libconfig.so: icinga::AExpression::Evaluate(boost::shared_ptr const&) const (+0x2e) [0x7f13db4b3f5e]
(12) libconfig.so: icinga::AExpression::OpDict(icinga::AExpression const*, boost::shared_ptr const&) (+0x248) [0x7f13db4b4498]
(13) libconfig.so: icinga::AExpression::Evaluate(boost::shared_ptr const&) const (+0x2e) [0x7f13db4b3f5e]
(14) libconfig.so: icinga::AExpression::OpDict(icinga::AExpression const*, boost::shared_ptr const&) (+0x248) [0x7f13db4b4498]
(15) libconfig.so: icinga::AExpression::Evaluate(boost::shared_ptr const&) const (+0x2e) [0x7f13db4b3f5e]
(16) libconfig.so: icinga::ConfigItem::GetProperties() (+0x336) [0x7f13db516c76]
(17) libconfig.so: icinga::ConfigItem::Commit() (+0x507) [0x7f13db5178c7]
(18) libicinga.so: icinga::Dependency::EvaluateApplyRuleOne(boost::shared_ptr const&, icinga::ApplyRule const&) (+0x118f) [0x7f13d6a018af]
(19) libicinga.so: icinga::Dependency::EvaluateApplyRule(icinga::ApplyRule const&) (+0xec1) [0x7f13d6a029d1]
(20) libicinga.so: void boost::_bi::list1 >::operator()(boost::_bi::type, void (*&)(icinga::ApplyRule const&), boost::_bi::list0&, int) (+0x55) [0x7f13d6a048b5]
(21) libicinga.so: boost::_bi::bind_t > >::operator()() (+0x3f) [0x7f13d6a0484f]
(22) libicinga.so: boost::detail::function::void_function_obj_invoker0 > >, void>::invoke(boost::detail::function::function_buffer&) (+0x1d) [0x7f13d6a0460d]
(23) libbase.so: boost::function0::operator()() const (+0x7b) [0x7f13dba87d9b]
(24) libbase.so: icinga::WorkQueue::WorkerThreadProc() (+0x5be) [0x7f13dbb605ae]
(25) libbase.so: boost::_mfi::mf0::operator()(icinga::WorkQueue*) const (+0x72) [0x7f13dbb632f2]
(26) libbase.so: void boost::_bi::list1 >::operator(), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) (+0x55) [0x7f13dbb63255]
(27) libbase.so: boost::_bi::bind_t, boost::_bi::list1 > >::operator()() (+0x3f) [0x7f13dbb631ef]
(28) libbase.so: boost::detail::thread_data, boost::_bi::list1 > > >::run() (+0x1c) [0x7f13dbb6319c]
(29) libboost_thread.so.1.55.0: (+0xc5ba) [0x7f13dc8285ba]
(30) libpthread.so.0: (+0x80a4) [0x7f13dbe840a4]
(31) libc.so.6: clone (+0x6d) [0x7f13d9a1704d]
[PN6icinga12ContextTraceE] =
(0) Evaluating 'apply' rule (in /etc/icinga2/conf.d/dns-dep.conf: 27:1-27:56)
(1) Evaluating 'apply' rules for Service 'localhost!http'
***
* This would indicate a runtime problem or configuration error. If you believe this is a bug in Icinga 2
* please submit a bug report at https://dev.icinga.org/ and include this stack trace as well as any other
* information that might be useful in order to reproduce this problem.
***
Aborted
Icinga 2 detected configuration errors. Check '/var/log/icinga2/startup.log' for details.
Updated by mfriedrich on 2014-09-09 21:50:40 +00:00
Updated by gbeutner on 2014-09-10 14:28:57 +00:00
Updated by mfriedrich on 2014-11-11 21:28:56 +00:00
Updated by mfriedrich on 2014-11-11 21:58:26 +00:00
Updated by mfriedrich on 2014-11-11 22:24:17 +00:00
Updated by mfriedrich on 2014-11-24 12:34:34 +00:00
Updated by mfriedrich on 2014-11-24 14:30:35 +00:00
Updated by gbeutner on 2015-01-07 08:27:09 +00:00
Updated by gbeutner on 2015-01-08 12:20:06 +00:00
Updated by mfriedrich on 2015-01-09 10:31:51 +00:00
Updated by mfriedrich on 2015-01-23 10:09:39 +00:00
Updated by mfriedrich on 2015-01-26 15:05:55 +00:00
Updated by gbeutner on 2015-01-30 14:50:48 +00:00
Updated by gbeutner on 2015-04-27 09:13:03 +00:00
Updated by mfriedrich on 2015-05-18 08:13:21 +00:00
This issue has been migrated from Redmine: https://dev.icinga.com/issues/6070
Created by gbeutner on 2014-04-29 08:28:30 +00:00
Assignee: (none) Status: Closed (closed on 2015-05-18 08:13:20 +00:00) Target Version: (none) Last Update: 2015-05-18 08:13:20 +00:00 (in Redmine)
a) Write exception info (stacktrace, context trace) to a separate log file. b) Make error messages more user-friendly and shorter
c) 'notice' level for connect messages d) Catch every exception which we are aware of. Other exceptions should still happen.
Most common error cases
Changesets
2014-05-12 08:27:01 +00:00 by gbeutner 9b7702195a060cb746c6f61c3c8b85b85d8eb467
2014-05-19 20:54:17 +00:00 by (unknown) 6c0b8db4bf4fbefe54b0a3c390ea5ad47e7b651d
2014-05-22 11:28:15 +00:00 by (unknown) dcd9c5b7e18329854a7c3cfe1d2cdbde3131ce3e
2014-05-22 11:47:38 +00:00 by (unknown) e7519981ea249a8c31261079214f66026e495dcc
2014-05-22 11:47:38 +00:00 by (unknown) 9e226947c86eb84a887f0c939a713967eace14e8
2014-05-22 14:07:57 +00:00 by (unknown) 7462c8320a678cc6ab55113a620de9eb93ba418c
2014-05-22 14:07:57 +00:00 by (unknown) aaa6154fd772a1ce95869f2b273fb2f9f65b441f
2014-05-22 14:07:57 +00:00 by (unknown) 0713918c5ec3abc32484e80e401e75d043c83d36
2014-05-22 14:07:57 +00:00 by (unknown) 4296ec3987aa2e83b240c24e21ebacbc379ef6bf
2014-05-22 19:20:18 +00:00 by (unknown) c2dadc5f88ba857b0171cf7d545dcbb66b1ee9e8
2014-05-22 21:47:03 +00:00 by mfriedrich 1df7518b358d42c7498001c577dd48730ae26f5f
2014-05-23 12:16:35 +00:00 by (unknown) 15f054b7fe888111da8cc50d52e75e42364c6eeb
2014-05-23 17:00:55 +00:00 by (unknown) 778b51ab6d9fd2d68133c2996780e6cdb63cf896
2014-05-23 17:07:44 +00:00 by (unknown) 820b1a340c63e7f62f46bc6eca5cf44356051a49
2014-06-05 12:36:50 +00:00 by (unknown) f1e77a057992eba48c59f6372e8aac67c598727e
2014-06-05 12:40:52 +00:00 by (unknown) bd916723f41ae9c97fe5e495a0166a5a21fd56dd
2014-06-05 13:03:56 +00:00 by (unknown) 47f19a2ce954a98772b8d08077e63a34c85896ba
2014-06-05 13:34:54 +00:00 by (unknown) 2989c0068000a9a9796a4625bed37883f5ed48db
2014-06-05 13:35:30 +00:00 by (unknown) 17668af2c290595ba731e366d03fc7a0d863ff2d
2014-06-05 13:45:37 +00:00 by (unknown) a416987031a7704621c2e5e7fbd097ad0aae4d33
2014-06-05 14:13:28 +00:00 by (unknown) 6a080edf80631e2df9a1335af199f2fb3df7c0e1
2014-06-05 14:17:53 +00:00 by (unknown) 0c021d94cb2a10dda1dfa4d3f61ce23f93a33ad0
2014-06-05 15:20:15 +00:00 by (unknown) f0b042025616d243c0eded4ee44a72764d2219b4
2014-06-05 15:43:34 +00:00 by (unknown) efa8fdcb8e5cebe08c27d13e5ba27998245a5063
2014-06-05 15:44:41 +00:00 by (unknown) 1064dc28fce1eb06ad485ce2664b4aa9280b36cf
2014-06-05 15:45:02 +00:00 by (unknown) 09ad04b09a135c8f5b56f50140e455ae47765c4c
2014-06-05 16:26:38 +00:00 by (unknown) 93992373be548428865bbfcafc1debfe8012f42d
2014-08-19 11:48:41 +00:00 by gbeutner 8485e0b94b49d7b0f1c7aec85a3bd7d071240012
2014-08-25 06:36:30 +00:00 by (unknown) f7f8bd9b00e1ca2dccdada1697e894c5f77fa315
2014-09-09 12:49:21 +00:00 by gbeutner 308ba451027cd01073d42d76aa909731c62dd1f6
2014-10-10 09:39:03 +00:00 by gbeutner 5962330b27df2c20c3de6665f0fc7fd38a5eff09
2014-12-20 14:29:04 +00:00 by (unknown) c433284d27fa61485b4687512f1a9cf000ab3ddb
2014-12-20 14:41:31 +00:00 by (unknown) 9f6b90cd2d14771ec0da967f9c10c76cf516ffb8
2015-01-29 15:51:47 +00:00 by (unknown) e0be9cd743206e4cb193cfcd60551bce4a301e44
2015-02-04 10:23:49 +00:00 by (unknown) 1d0483d41f71ad330098ca0f09e76b49b3249fb3
Relations: