search

dotnet / dotnet-monitor

This repository contains the source code for .NET Monitor - a tool that allows you to gather diagnostic data from running applications using HTTP endpoints
MIT License
617 stars 103 forks source link

Managed Prometheus scraping in Aks has timestamp error #4469

Open wiktork opened 1 year ago

wiktork commented 1 year ago
  1. Enable Managed Prometheus integration in an Aks Cluster.
  2. Follow https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/prometheus-metrics-scrape-configuration#configmaps to create a pod level scraper against dotnet-monitor's /metrics endpoint.
  3. Check the prometheus targets using https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/prometheus-metrics-troubleshoot#prometheus-interface

Notice that the dotnet-monitor target has the following error:

Note that the error repros with a 30s or 1m scraping interval and a 5-second metrics interval.

Workaround: Configure Metrics__Count to be 1.

ajay-sainy commented 1 year ago

Having same issue. I had to set Metrics__Count to 1 and scrape interval to 5s