webdevops / azure-metrics-exporter

Azure Monitor metrics exporter for Prometheus with dimension support, template engine and ServiceDiscovery
MIT License
124 stars 25 forks source link

Fetching tags of Resourcetype #40

Open Amrish-Sharma opened 1 year ago

Amrish-Sharma commented 1 year ago

Hello mblaschke,

Thanks much for exporter, I am able to export metrics of PostgreSQL for Flexible Server but getting metrics like

azure_metric_postgres_active_connections_count{aggregation="average",interval="PT5M",resourceGroup="mco_adia_prd",resourceID="/subscriptions/{subs_id}/resourcegroups/{rg_name}/providers/microsoft.dbforpostgresql/flexibleservers/{db_name}",resourceName="{resource_name}",subscriptionID={sub_id}",tag_owner="",timespan="PT5M"} 91

I have deployed this exporter on my central monitoring server where Prometheus server is running, I would like to fetch the tags of PostgreSQL instance to relabel config so that I'll be able to set alerts with distinguished clients, environment. Is there anyway I can do that in azure-metrics-exporter.

My current Scrape config is


- job_name: 'azure-metrics-postgres'
  scrape_timeout: '15s'
  metrics_path: '/probe/metrics/list'
  scrape_interval: 1m
  params:
    name: ["azure_metric_postgres"]
    template: ["{name}_{metric}_{unit}"]
    help: ["Custom help with {metric}"]
    subscription:
    - 'sub_1'
    - 'sub_2'
    - 'sub_3'
    resourceType: ["Microsoft.DBforPostgreSQL/flexibleServers"]
    metric:
    - disk_queue_depth
    - active_connections
    - cpu_percent
    - storage_used
    - storage_free
    - deadlocks
    - connections_failed
    - memory_percent
    - storage_percent
    interval: ["PT5M"]
    timespan: ["PT5M"]
    aggregation:
    - average
    metricTop: ["10"]
  static_configs:
    - targets: ["localhost:2100"]
Amrish-Sharma commented 1 year ago

I have created systemd service as below where I am trying to fetch the tags of resource owner(which is already fetched by default,

Other tags such as Customer, Env, Project, Name is not getting fetched


[Unit]
Description=Prometheus Azure Metrics Exporter
After=network.target

[Service]
Type=simple
User=am_exp
Group=am_exp
Nice=-5
SyslogIdentifier=prometheus-azure-metrics-exporter
Restart=always
ExecStart=/usr/local/bin/azure_metrics_exporter \
  --log.debug \
  --log.trace \
  --azure.resource-tag=owner customer env project name \
  --server.bind=0.0.0.0:2100

[Install]
WantedBy=multi-user.target
eberkut commented 6 months ago

I'm also interested in the same feature. My understanding is that --azre.resource-tag is actually used for service discovery. On the other hand the following lead me to believe this tags should automatically be added as labels by default: https://github.com/webdevops/azure-metrics-exporter#resourcetags-handling https://github.com/webdevops/azure-metrics-exporter/blob/main/metrics/insights.target.go#L67 https://github.com/webdevops/go-common/blob/main/azuresdk/README.md#tag-manager

Is that correct? Is there any additional configuration? For the record, I'm not seeing any of my resources custom tags being added as labels in my configuration, e.g. with this:

- job_name: az-keyvault
  static_configs:
    - targets:
        - "localhost:8080"
  scrape_interval: 1m
  metrics_path: /probe/metrics/list
  params:
    name: ["azure_metric_keyvault"]
    subscription:
      - ${subscription_id}
    filter: ["resourceType eq 'Microsoft.KeyVault/vaults'"]
    metric:
      - Availability
      - SaturationShoebox
      - ServiceApiHit
      - ServiceApiLatency
      - ServiceApiResult
    aggregation:
      - Average
      - Count
    interval: ["PT1M"]
    timespan: ["PT1M"]
    dimensions:
      - ActivityType
      - ActivityName
      - StatusCode
      - StatusCodeClass