grafana / alloy

OpenTelemetry Collector distribution with programmable pipelines
https://grafana.com/oss/alloy
Apache License 2.0
1.38k stars 201 forks source link

Alloy windows client spams eventlog when loosing contact with remote #1730

Open davidtannfors opened 1 month ago

davidtannfors commented 1 month ago

Request

Hello

When there's a network glitch or our remote service is down for a short while Alloy spams the windows eventlog like crazy. Could there be some kind of wait and retry period implemented to avoid this?

error messages like this every second: {"ts":"2024-08-20T08:32:01.9832581Z","level":"error","msg":"non-recoverable error","component_path":"/","component_id":"prometheus.remote_write.integrations","subcomponent":"rw","remote_name":"xxxx","url":"xxxx/metrics/api/v1/push","count":2000,"exemplarCount":0,"err":"server returned HTTP status 400 Bad Request: user=xxxx: err: out of order sample. timestamp=2024-08-20T08:21:30.698Z, series={name=\"windows_memory_swap_page_operations_total\", agent_hostname=\"xxxxx\", instance=\"xxxxx\", job=\"integrations/windows_exporter\"}"}

Use case

To avoid spamming the eventlog

github-actions[bot] commented 5 hours ago

This issue has not had any activity in the past 30 days, so the needs-attention label has been added to it. If the opened issue is a bug, check to see if a newer release fixed your issue. If it is no longer relevant, please feel free to close this issue. The needs-attention label signals to maintainers that something has fallen through the cracks. No action is needed by you; your issue will be kept open and you do not have to respond to this comment. The label will be removed the next time this job runs if there is new activity. Thank you for your contributions!