webdevops / azure-metrics-exporter

Azure Monitor metrics exporter for Prometheus with dimension support, template engine and ServiceDiscovery
MIT License
122 stars 25 forks source link

Could not Collect "Stats", "Deployment" and "Release" #96

Open dadoMT opened 1 month ago

dadoMT commented 1 month ago

Hi,

I've recently setup this one in our K8s Cluster and found out some issues

As you can see here, there are some metrics (specifically build and agent metrics) that I can collect

k logs -n monitoring azure-devops-exporter-77ffb5d55b-4vft7 | tail -n 50 2024-07-22T10:07:14.282Z info collector/collector.go:308 starting metrics collection {"collector": "Project"} 2024-07-22T10:07:14.282Z info collector/collector.go:339 finished metrics collection, next run in 30s {"collector": "Project", "duration": 0.000093791, "nextRun": "2024-07-22T10:07:44.282Z"} 2024-07-22T10:07:35.553Z info collector/collector.go:308 starting metrics collection {"collector": "LatestBuild"} 2024-07-22T10:07:36.597Z info collector/collector.go:339 finished metrics collection, next run in 30s {"collector": "LatestBuild", "duration": 1.043513255, "nextRun": "2024-07-22T10:08:06.597Z"} 2024-07-22T10:07:40.276Z info collector/collector.go:308 starting metrics collection {"collector": "AgentPool"} 2024-07-22T10:07:43.409Z info collector/collector.go:339 finished metrics collection, next run in 30s {"collector": "AgentPool", "duration": 3.133141156, "nextRun": "2024-07-22T10:08:13.409Z"}

however, I'm having problem getting some metrics, specifically "Release" or "Deployment" metrics, "Stats" would be cool to have as well

k logs -n monitoring azure-devops-exporter-77ffb5d55b-4vft7 | grep Sandbox | tail -n 100 2024-07-22T08:25:42.525Z error azure-devops-exporter/metrics_stats.go:187 response status code is 404 (expected 200), url: https://dev.azure.com/<redacted>/<redacted-2>/_apis/release/releases?api-version=5.1&isDeleted=false&$expand=94&minCreatedTime=2024-07-22T07%3A55%3A42Z&$top=100&queryOrder=descending {"collector": "Stats", "project": "Sandbox"} 2024-07-22T08:25:43.072Z error azure-devops-exporter/metrics_deployment.go:81 response status code is 404 (expected 200), url: https://dev.azure.com/<redacted>/<redacted-2>/_apis/release/definitions?api-version=5.1&isDeleted=false&$top=100&$expand=environments,lastRelease {"collector": "Deployment", "project": "Sandbox"} 2024-07-22T08:25:43.089Z error azure-devops-exporter/metrics_release.go:176 response status code is 404 (expected 200), url: https://dev.azure.com/<redacted>/<redacted-2>/_apis/release/definitions?api-version=5.1&isDeleted=false&$top=100&$expand=environments,lastRelease {"collector": "Release", "project": "Sandbox"} 2024-07-22T08:55:43.861Z error azure-devops-exporter/metrics_deployment.go:81 response status code is 404 (expected 200), url: https://dev.azure.com/<redacted>/<redacted-2>/_apis/release/definitions?api-version=5.1&isDeleted=false&$top=100&$expand=environments,lastRelease {"collector": "Deployment", "project": "Sandbox"} 2024-07-22T08:55:43.876Z error azure-devops-exporter/metrics_release.go:176 response status code is 404 (expected 200), url: https://dev.azure.com/<redacted>/<redacted-2>/_apis/release/definitions?api-version=5.1&isDeleted=false&$top=100&$expand=environments,lastRelease {"collector": "Release", "project": "Sandbox"} 2024-07-22T08:55:44.199Z error azure-devops-exporter/metrics_stats.go:187 response status code is 404 (expected 200), url: https://dev.azure.com/<redacted>/<redacted-2>/_apis/release/releases?api-version=5.1&isDeleted=false&$expand=94&minCreatedTime=2024-07-22T08%3A25%3A44Z&$top=100&queryOrder=descending {"collector": "Stats", "project": "Sandbox"} Here's what I see in prometheus

image

Here's grafana that I'm currently building

image

Could you please point me out to the actual metrics documentation? I want to know which metrics can be used for time series panel.

For example I want to see which pipelines failed on a specific time range, etc.. how many where run.

Please help out, the main issue is collecting "Releases" or "Deployment" metrics, that would be very helpful on the dashboard I'm creating for my team

dadoMT commented 1 month ago

is there anyone that can help on this one?