Icinga / icinga2

The core of our monitoring platform with a powerful configuration language and REST API.
https://icinga.com/docs/icinga2/latest
GNU General Public License v2.0
2.03k stars 578 forks source link

Windows Service not starting when using Service Account #6266

Closed matthenning closed 6 years ago

matthenning commented 6 years ago

I'm trying to setup Icinga 2 Agents on Windows to use a domain account but I can't get the service to start. I suspect a permission problem but I can't get ahold of it.

What I've tried:

Telegraf and Winlogbeat are already running successfully under the new service account.

Expected Behavior

The service starts when using a different account than the default account.

Current Behavior

The service crashes on startup:

grafik

There is no crash file written and no entries in either icinga2.log or debug.log. The Windows Application Log does not offer any details either.

Steps to Reproduce (for bugs)

Your Environment

============== GENERAL INFORMATION ==============

Application version: v2.8.4
Installation root: C:\Program Files\ICINGA2\
Sysconf directory: C:\ProgramData\icinga2\etc
Run directory: C:\ProgramData\icinga2\var\run
Local state directory: C:\ProgramData\icinga2\var
Package data directory: C:\Program Files\ICINGA2\\share\icinga2
State path: C:\ProgramData\icinga2\var/lib/icinga2/icinga2.state
Objects path: C:\ProgramData\icinga2\var/cache/icinga2/icinga2.debug
Vars path: C:\ProgramData\icinga2\var/cache/icinga2/icinga2.vars
PID path: C:\ProgramData\icinga2\var\run/icinga2/icinga2.pid

Enabled features: api checker debuglog mainlog Disabled features: command compatlog elasticsearch gelf graphite ido-mysql ido-pgsql influxdb livestatus notification opentsdb perfdata statusdata ============== OBJECT INFORMATION ==============

Checking object file from C:\ProgramData\icinga2\var/cache/icinga2/icinga2.debug Found the 235 objects: Type : Count ApiListener : 1 CheckCommand : 213 CheckerComponent : 1 Endpoint : 2 FileLogger : 1 HostGroup : 9 IcingaApplication : 1 NotificationCommand : 1 TimePeriod : 2 Zone : 4

The objects origins are:

C:\Program Files\ICINGA2\share\icinga2\include/command-icinga.conf C:\Program Files\ICINGA2\share\icinga2\include/command-nscp-local.conf C:\Program Files\ICINGA2\share\icinga2\include/command-plugins-manubulon.conf C:\Program Files\ICINGA2\share\icinga2\include/command-plugins-windows.conf C:\Program Files\ICINGA2\share\icinga2\include/command-plugins.conf C:\Program Files\ICINGA2\share\icinga2\include/plugins-contrib.d/databases.conf C:\Program Files\ICINGA2\share\icinga2\include/plugins-contrib.d/hardware.conf C:\Program Files\ICINGA2\share\icinga2\include/plugins-contrib.d/icingacli.conf C:\Program Files\ICINGA2\share\icinga2\include/plugins-contrib.d/ipmi.conf C:\Program Files\ICINGA2\share\icinga2\include/plugins-contrib.d/logmanagement.conf C:\Program Files\ICINGA2\share\icinga2\include/plugins-contrib.d/metrics.conf C:\Program Files\ICINGA2\share\icinga2\include/plugins-contrib.d/network-components.conf C:\Program Files\ICINGA2\share\icinga2\include/plugins-contrib.d/network-services.conf C:\Program Files\ICINGA2\share\icinga2\include/plugins-contrib.d/operating-system.conf C:\Program Files\ICINGA2\share\icinga2\include/plugins-contrib.d/raid-controller.conf C:\Program Files\ICINGA2\share\icinga2\include/plugins-contrib.d/smart-attributes.conf C:\Program Files\ICINGA2\share\icinga2\include/plugins-contrib.d/storage.conf C:\Program Files\ICINGA2\share\icinga2\include/plugins-contrib.d/virtualization.conf C:\Program Files\ICINGA2\share\icinga2\include/plugins-contrib.d/vmware.conf C:\Program Files\ICINGA2\share\icinga2\include/plugins-contrib.d/web.conf C:\ProgramData\icinga2\etc\icinga2/zones.conf C:\ProgramData\icinga2\etc\icinga2\features-enabled..\features-available/api.conf C:\ProgramData\icinga2\etc\icinga2\features-enabled..\features-available/checker.conf C:\ProgramData\icinga2\etc\icinga2\features-enabled..\features-available/mainlog.conf C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/commands.conf C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/hostgroups.conf C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/timeperiods.conf

============== LOGS AND CRASH REPORTS ==============

Getting the last 20 lines of 1 FileLogger objects. Logger main-log at path: C:\ProgramData\icinga2\var/log/icinga2/icinga2.log [begin: 'C:\ProgramData\icinga2\var/log/icinga2/icinga2.log' line: 7299]

[2018-04-27 08:24:54 +0200] information/WorkQueue: #23 (JsonRpcConnection, #15) items: 0, rate: 0/s (0/min 0/5min 0/15min);

[2018-04-27 08:24:54 +0200] information/WorkQueue: #9 (JsonRpcConnection, #1) items: 0, rate: 0/s (0/min 0/5min 0/15min);

[2018-04-27 08:24:54 +0200] information/WorkQueue: #14 (JsonRpcConnection, #6) items: 0, rate: 0/s (0/min 0/5min 0/15min);

[2018-04-27 08:24:54 +0200] information/WorkQueue: #21 (JsonRpcConnection, #13) items: 0, rate: 0/s (0/min 0/5min 0/15min);

[2018-04-27 08:24:54 +0200] information/WorkQueue: #25 (JsonRpcConnection, #17) items: 0, rate: 0/s (0/min 0/5min 0/15min);

[2018-04-27 08:24:54 +0200] information/WorkQueue: #10 (JsonRpcConnection, #2) items: 0, rate: 0/s (0/min 0/5min 0/15min);

[2018-04-27 08:24:54 +0200] information/WorkQueue: #16 (JsonRpcConnection, #8) items: 0, rate: 0/s (0/min 0/5min 0/15min);

[2018-04-27 08:24:54 +0200] information/WorkQueue: #12 (JsonRpcConnection, #4) items: 0, rate: 0/s (0/min 0/5min 0/15min);

[2018-04-27 08:24:54 +0200] information/WorkQueue: #17 (JsonRpcConnection, #9) items: 0, rate: 0/s (0/min 0/5min 0/15min);

[2018-04-27 08:24:54 +0200] information/WorkQueue: #13 (JsonRpcConnection, #5) items: 0, rate: 0/s (0/min 0/5min 0/15min);

[2018-04-27 08:24:54 +0200] information/WorkQueue: #18 (JsonRpcConnection, #10) items: 0, rate: 0/s (0/min 0/5min 0/15min);

[2018-04-27 08:24:54 +0200] information/WorkQueue: #15 (JsonRpcConnection, #7) items: 0, rate: 0/s (0/min 0/5min 0/15min);

[2018-04-27 08:24:54 +0200] information/WorkQueue: #19 (JsonRpcConnection, #11) items: 0, rate: 0/s (0/min 0/5min 0/15min);

[2018-04-27 08:24:54 +0200] information/WorkQueue: #22 (JsonRpcConnection, #14) items: 0, rate: 0/s (0/min 0/5min 0/15min);

[2018-04-27 08:24:54 +0200] information/WorkQueue: #24 (JsonRpcConnection, #16) items: 0, rate: 0/s (0/min 0/5min 0/15min);

[2018-04-27 08:24:54 +0200] information/WorkQueue: #26 (JsonRpcConnection, #18) items: 0, rate: 0/s (0/min 0/5min 0/15min);

[2018-04-27 08:24:54 +0200] information/WorkQueue: #27 (JsonRpcConnection, #19) items: 0, rate: 0/s (0/min 0/5min 0/15min);

[2018-04-27 08:25:40 +0200] warning/PluginCheckTask: Check command for object '***' (PID: 0, arguments: '"C:\Program Files\ICINGA2\/sbin/check_dummy" 0 "Check was successful."') terminated with exit code 127, output: Command "C:\Program Files\ICINGA2\/sbin/check_dummy" 0 "Check was successful." failed to execute: 2, "The system cannot find the file specified."

[2018-04-27 08:26:38 +0200] warning/PluginCheckTask: Check command for object '***' (PID: 0, arguments: '"C:\Program Files\ICINGA2\/sbin/check_dummy" 0 "Check was successful."') terminated with exit code 127, output: Command "C:\Program Files\ICINGA2\/sbin/check_dummy" 0 "Check was successful." failed to execute: 2, "The system cannot find the file specified."

[2018-04-27 08:27:37 +0200] warning/PluginCheckTask: Check command for object '***' (PID: 0, arguments: '"C:\Program Files\ICINGA2\/sbin/check_dummy" 0 "Check was successful."') terminated with exit code 127, output: Command "C:\Program Files\ICINGA2\/sbin/check_dummy" 0 "Check was successful." failed to execute: 2, "The system cannot find the file specified."

[end: 'C:\ProgramData\icinga2\var/log/icinga2/icinga2.log' line: 7319]

No crash logs found in C:\ProgramData\icinga2\var/log/icinga2/crash/

============== CONFIGURATION FILES ==============

A collection of important configuration files follows, please make sure to remove any sensitive data such as credentials, internal company names, etc [begin: 'C:\ProgramData\icinga2\etc/icinga2/icinga2.conf']

/**

* Icinga 2 configuration file

* - this is where you define settings for the Icinga application including

* which hosts/services to check.

*

* For an overview of all available configuration options please refer

* to the documentation that is distributed as part of Icinga 2.

*/

/**

* The constants.conf defines global constants.

*/

include "constants.conf"

/**

* The zones.conf defines zones for a cluster setup.

* Not required for single instance setups.

*/

include "zones.conf"

/**

* The Icinga Template Library (ITL) provides a number of useful templates

* and command definitions.

* Common monitoring plugin command definitions are included separately.

*/

include

include

include

include

/**

* This includes the Icinga 2 Windows plugins.

*/

include

/**

* This includes the NSClient++ check commands.

*/

include

/**

* The features-available directory contains a number of configuration

* files for features which can be enabled and disabled using the

* icinga2 feature enable / icinga2 feature disable CLI commands.

* These commands work by creating and removing symbolic links in

* the features-enabled directory.

*/

include "features-enabled/*.conf"

/**

* Although in theory you could define all your objects in this file

* the preferred way is to create separate directories and files in the conf.d

* directory. Each of these files must have the file extension ".conf".

*/

include_recursive "conf.d"

[end: 'C:\ProgramData\icinga2\etc/icinga2/icinga2.conf'] [begin: 'C:\ProgramData\icinga2\etc/icinga2/zones.conf']

/*

* Generated by Icinga 2 node setup commands

* on 2018-04-26 12:49:54 +0200

*/

object Endpoint "***" {

host = "***"

port = "5665"

}

object Zone "master" {

endpoints = [ "***" ]

}

object Zone "global-templates" {

global = true

}

object Zone "director-global" {

global = true

}

object Endpoint NodeName {

}

object Zone ZoneName {

endpoints = [ NodeName ]

parent = "master"

}

[end: 'C:\ProgramData\icinga2\etc/icinga2/zones.conf']

==================================================== Finished collection at 2018-04-27 08:47:18 Took 0.156000 seconds


* Enabled features (`icinga2 feature list`): 

Disabled features: command compatlog elasticsearch gelf graphite ido-mysql ido-pgsql influxdb livestatus notification opentsdb perfdata statusdata Enabled features: api checker debuglog mainlog


* Config validation (`icinga2 daemon -C`):

information/cli: Icinga application loader (version: v2.8.4) information/cli: Loading configuration file(s). information/ConfigItem: Committing config item(s). information/ApiListener: My API identity: **** warning/ApplyRule: Apply rule 'System - Load' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 1:0-1:28) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'System - Memory' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 9:1-9:31) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'App - Icinga 2 Health' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 17:1-17:37) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'App - Icinga 2 Cluster' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 25:1-25:38) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'App - Icinga 2 Cluster Zone' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 33:1-33:43) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'System - Disk ' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 43:1-43:74) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'App - Icinga 2 Service' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 52:1-52:38) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'App - InfluxDB Process' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 60:1-60:38) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'App - InfluxDB Port' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 68:1-68:35) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'App - Icinga 2 Port' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 76:1-76:35) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'System - Swap' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 84:1-84:29) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'System - CPU' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 92:1-92:28) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'System - Disk ' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 100:1-100:74) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'System - Memory' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 109:1-109:31) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'System - Pagefile' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 117:1-117:33) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'System - Uptime' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 125:1-125:31) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'System - Uptime' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 133:1-133:31) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'System - Network' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 141:1-141:32) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'System - Network' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 149:1-149:32) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'System - Cron' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 157:1-157:29) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'System - Task Scheduler' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 165:1-165:39) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'System - Updates' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 173:1-173:32) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'System - Updates' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 181:1-181:32) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'System - Filebeat' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 189:1-189:33) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'App - MySQL Health' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 197:1-197:34) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'App - Logstash Health' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 205:1-205:37) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'App - Logstash Health' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 213:1-213:37) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'App - Elasticsearch Health' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 221:1-221:42) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'App - Elasticsearch Health' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 229:1-229:42) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'App - NGINX Process' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 237:1-237:35) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'App - Icingaweb 2 Status' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 245:1-245:40) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'App - Grafana Status' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 253:1-253:36) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'App - Kibana Status' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 261:1-261:35) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'System - Telegraf' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 269:1-269:33) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'App - Logstash Process' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 277:1-277:38) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'App - Elasticsearch Process' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 285:1-285:43) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'App - Icinga 2 Cluster Zone' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 293:1-293:43) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'App - InfluxDB Health' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 302:1-302:37) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'App - IQ Port' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 310:1-310:29) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'App - IQ Process' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 318:1-318:32) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'System - Memory' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 326:1-326:31) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'App - Icinga 2' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 334:1-334:30) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'App - Icinga 2 Service' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 342:1-342:38) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'System - Telegraf' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 350:1-350:33) for type 'Service' does not match anywhere! warning/ApplyRule: Apply rule 'App - MSSQL Service' (in C:\ProgramData\icinga2\var\lib\icinga2\api\zones/director-global/director/service_apply.conf: 358:1-358:35) for type 'Service' does not match anywhere! information/ConfigItem: Instantiated 1 ApiListener. information/ConfigItem: Instantiated 4 Zones. information/ConfigItem: Instantiated 2 FileLoggers. information/ConfigItem: Instantiated 2 Endpoints. information/ConfigItem: Instantiated 2 TimePeriods. information/ConfigItem: Instantiated 9 HostGroups. information/ConfigItem: Instantiated 1 IcingaApplication. information/ConfigItem: Instantiated 213 CheckCommands. information/ConfigItem: Instantiated 1 NotificationCommand. information/ConfigItem: Instantiated 1 CheckerComponent. information/ScriptGlobal: Dumping variables to file 'C:\ProgramData\icinga2\var/cache/icinga2/icinga2.vars' information/cli: Finished validating the configuration file(s).

Crunsher commented 6 years ago

Thank you for opening this issue, we have yet to investigate and reproduce this. If anybody else is having this problem, please don't hesitate to comment here.

Could you take a look at your EventViewer and see if it has a more informative error message?

matthenning commented 6 years ago

There are 2 informational events in the System log, but no errors:

The Icinga 2 service entered the running state. 
The Icinga 2 service entered the stopped state. 
dnsmichi commented 6 years ago

What's different between the domain account and the local account? I'm not a Windows guy, but I could imagine that group policies apply which disallow certain "root" administrative permissions such a service would need.

In comparison to that, are you able to run NSClient++ with that domain account?

matthenning commented 6 years ago

To make sure it isn't related to a GPO I've created a local user, granted the permissions mentioned above, and tried to start the service. The result was the same.

NSClient++ is starting with both the local and the domain account without granting any special permissions.

Crunsher commented 6 years ago

Did you use the agent wizard to install the service for your user? I sadly am not well versed in Windows access control either :/

matthenning commented 6 years ago

I ran the msi and then the node wizard to setup the cluster. I'm not certain whether it's a permission problem. Does Icinga require permission on other directories than mentioned in the issue?

Are there any other debug options to see what's going on before the service crashes?

Crunsher commented 6 years ago

There are neither crash reports not is the log very informative.What happens when you run icinga2 in the foreground as that user? I don't think there are any other directories Icinga needs permissions on to run

thammw commented 6 years ago

Please try to give full access to "C:\ProgramData\icinga2\etc" Delegation seems to be deactivated for this folder. Should work after that... :)

matthenning commented 6 years ago

Yep, that's what I just noticed, too @thammw. And that's the solution as well. Thank you all.