roobre
commented
3 years ago Hi, thanks for reporting this issue. We are aware of the winservices integration not behaving properly in situations of high CPU and/or memory pressure. To our knowledge, the root cause for this is often WMI, a windows API the integration used and that is known to not work reliably in this situations.
Could you provide the lines that appear after the last 10 lines out of 7476 message? This would help us identify if this is the actual issue.
CharlieWinters
mangulonr
davidgit
The integration is being used on a Microsoft Windows Server 2016 Datacenter host. At certain times the host seems to get maxed out on resources, i.e. we get CPU, load average, disk utilisation spikes, and then the integration appears to stop sending samples.
The issue is that the integration doesn't appear to recover until the Infrastructure agent is restarted.
Description
Some internal info has been outlined in support ticket #448676
We get errors in the logs like: time="2021-02-27T08:21:39Z" level=warning msg="integration exited with error state" component=integrations.runner.Runner error="exit status 2" integration_name=com.newrelic.winservices stderr="(last 10 lines out of 7476): ....
and a message below for the monitored services: time="2021-02-28T15:01:42Z" level=info msg="Removing inventory cache" agentEntityIDChanged=false component=PatchSender entityKey=":" offlineTime=24h0m0s
Expected Behavior
I would expect the integration to recover once resources become available again.
NR Diag results
Steps to Reproduce
Max out resources on the host, observe windows services samples not being sent, integration does not recover.
Your Environment
Microsoft Windows Server 2016
Additional context
I've built a troubleshooting dashboard that is detailed in support ticket: #448676.