search

prometheus-community / windows_exporter

Prometheus exporter for Windows machines
MIT License
2.94k stars 704 forks source link

Version 0.29.1 issue with monitoring services #1669

Closed andonovski closed 1 month ago

andonovski commented 1 month ago

Current Behavior

I don't know if it's really a bug or something with my configuration. In version 0.29.1 I get alerts for stopped services whose start type is Automatic Triggered. In version 0.27.2 this is not happening. I did upgrade from0.27.2 to 0.29.1

On my machines there are 3 services that are stopped and start type is Automatic Triggered:

Now, for Software protection there is no alert, for other two I have alert. Server restart or windows exporter service restart doesn't help.

Expected Behavior

Expected behavior is that there are no alerts for services that are stopped, and they have start type Automatic, Triggered. This was the case in 0.27.2, which I can confirm, I still have one machine with this older version.

Steps To Reproduce

No response

Environment

windows_exporter logs

There are no logs for this, just very different behavior from previous version.

Anything else?

No response

JDA88 commented 1 month ago

Can you please post the alerting rule that trigger?

andonovski commented 1 month ago

This is alert rule that triggers the alert: (windows_service_state{name!~"remoteregistry|sppsvc|edgeupdate|staterepository|dosvc",state="stopped"} == 1) * on (name, instance) windows_service_start_mode{start_mode="auto"} == 1

Now that I look at this, it is strange that I am getting alerts only for two services which should be excluded. It seems my alerting rule is suddenly wrong.

Now, for windows exporter 0.27.2, no alert for these services. For 0.29.1, I have alerts for these services. Strange behavior, I haven't seen any change for metrics for services. Maybe I missed something or my alerting rule is bad from the start but it was working somehow.

JDA88 commented 1 month ago

Ho ok, you added the windows_service_start_mode to the query. I can't test this atm but #1584 refactored the service collector in 0.29.0, look at the row metrics to see if one of the changes affected your query

andonovski commented 1 month ago

To answer to myself, and anyone else who might have same issue, now the service names are not all small caps. In Prometheus the service names are now like this: old -> remoteregistry is now RemoteRegistry old-> dosvc is now DoSvc

So, now it matches exactly how the service is seen in Windows, small and capital letters. I can't find in any change that this is now new behavior.

JDA88 commented 1 month ago

Ho! damn sorry I found this one also and forgot about it! You are right and in fact is not documented!

A good way to deal with it and have the alert compatible with both versions is to have the RegEx case insensitive (?i)

jkroepke commented 1 month ago

I have added this to the documentation change (https://github.com/prometheus-community/windows_exporter/pull/1584).

Is there an issue left to mention?

andonovski commented 1 month ago

No, no other issues. Thank you very much foe thw help!

On Sat, Oct 5, 2024, 10:38 Jan-Otto Kröpke @.***> wrote:

I have added this to the documentation change (#1584 https://github.com/prometheus-community/windows_exporter/pull/1584).

Is there an issue left to mention?

— Reply to this email directly, view it on GitHub https://github.com/prometheus-community/windows_exporter/issues/1669#issuecomment-2394982425, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASPGAGVURJNXZBG5GWBA7GLZZ6QPTAVCNFSM6AAAAABPNBSXV6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOJUHE4DENBSGU . You are receiving this because you authored the thread.Message ID: @.***>