webdevops / azure-metrics-exporter

Azure Monitor metrics exporter for Prometheus with dimension support, template engine and ServiceDiscovery
MIT License
118 stars 24 forks source link

How can I support this filter by RateLimitKey? #62

Open callumj opened 10 months ago

callumj commented 10 months ago

I want to break down BlockedCalls and Latency for a CognitiveServices metric by RateLimitKey which I have confirmed is possible via the ARM library:

res, err := client.List(ctx, "subscriptions/<SUB_ID>/resourceGroups/<GROUP>/providers/Microsoft.CognitiveServices/accounts/<ACCOUNT>", &armmonitor.MetricsClientListOptions{
        Interval:        to.Ptr("PT1M"),
        Metricnames:     to.Ptr("BlockedCalls,Latency"),
        Aggregation:     to.Ptr("Average"),
        Filter:          to.Ptr("RatelimitKey eq '*'"),
        ResultType:      nil,
        Metricnamespace: to.Ptr("Microsoft.CognitiveServices/accounts"),
    })

Translating this into:

- job_name: azure-metrics-cognitive-accounts-blocked-calls
  scrape_interval: 30s
  metrics_path: /probe/metrics/scrape
  params:
    name: ["microsoft_cognitive_services_accounts_blocked_calls"]
    subscription:
      - "<SUB_ID>"
    target:
      - "subscriptions/<SUB_ID>/resourceGroups/<GROUP>/providers/Microsoft.CognitiveServices/accounts/<ACCOUNT>",
    metricNamespace: ["Microsoft.CognitiveServices/accounts"]
    filter: ["RatelimitKey eq '*'"]
    metric:
    - BlockedCalls
    interval: ["PT1M"]
    timespan: ["PT1M"]
    aggregation:
    - total
  static_configs:
  - targets: ["localhost:8080"]

ends up with an error

{
  "error": {
    "code": "InvalidFilterInQueryString",
    "message": "Invalid $filter 'RatelimitKey eq '*'' specified in the query string."
  }
}

Presumably because it first wants to scrape the subscription list using FindSubscriptionResources. But I should not need that right?

mblaschke commented 10 months ago

can you try it with metricTop: ["10"]? so you get the top 10 from the filter?

it's difficult to find the correct values, but it helps you to query the metrics via Azure portal and use the browser console to check what values are sent.

mblaschke commented 10 months ago

if you only want to get metrics from one resource you should use /probe/metrics/resource