mr-karan / aws-dc-exporter

Prometheus metrics exporter for AWS Direct Connect
MIT License
1 stars 2 forks source link

Tag labelling #2

Open vladsamoylik opened 3 years ago

vladsamoylik commented 3 years ago

Good day! I have a couple of questions:

  1. How can I receive information from two or more regions at the same time by one aws-dc-exporter installation?
  2. How can I request the values of aws_dc_connections and aws_dc_bgp_peers labels.
mr-karan commented 3 years ago
  1. You can do that by creating multiple jobs for different regions. Here's an example:
[[app.jobs]]
name="region-a" # Unique Job Name
[app.jobs.aws_creds]
region="ap-south-1" # Required.
[[app.jobs]]
name="region-b" # Unique Job Name
[app.jobs.aws_creds]
region="-us-west-1" # Required.
  1. Sorry, I didn't get you. You could make queries with aws_dc_connections{} / aws_dc_bgp_peers{} on PromQL. If you wanna see all the labels that are exported, just make this empty query on Prometheus, it'll show a list of labels.
vladsamoylik commented 3 years ago

Thanks! I got an exhaustive answer to the first question, and it works. But with regard to the second, not everything is so simple. Our task is to match bgp_peer_id with aws_dc_virtual_interfaces in order to exclude the firing of an alert on bgp_peer_id for those connections that do not work as they are in configuration mode.

Unfortunately, I cannot get the following tree by giving a promql request: aws_dc_virtual_interfaces> bgp_peer_id. If this is a long time for you, then I can try to increase this functionality on my own and send you a fix.

vladsamoylik commented 3 years ago

And yet, so I corrected the expression for the alert manager, the ones described for you do not work correctly for me:

  - alert: AWSDCConnectionDown
    expr: count(aws_dc_connections{conn_state="down", conn_name!~".*dev"}) by (conn_id, conn_name) > 0
    for: 0m
    labels:
      severity: 4
    annotations:
      title: "AWS DC Connection down (Connection name:  {{ $labels.conn_id }}, Connection ID: {{ $labels.conn_id }})."
  - alert: AWSBGPPeerDown
    expr: count(aws_dc_bgp_peers{bgp_status="down"}) by (bgp_peer_id) > 0
    for: 0m
    labels:
      severity: 4
    annotations:
      title: "AWS DC BGP Peer down (Peer ID: {{ $labels.bgp_peer_id }})."

Thanks!

vladsamoylik commented 3 years ago

Exported data of a couple of my PromQL queries:

aws_dc_virtual_interfaces{virt_interface_id="dxvif-XXXXXXXX"}
aws_dc_virtual_interfaces{customer_address="0.0.0.0/30", exported_job="awd-dc", instance="localhost:9980", job="aws-dc", location="Location", service="direct-connect", virt_interface_id="dxvif-XXXXXXX", virt_interface_name="Name", virt_interface_state="available"}

aws_dc_bgp_peers{bgp_peer_id="dxpeer-XXXXXXXX"}
aws_dc_bgp_peers{aws_device_v2="DevName", bgp_peer_id="dxpeer-XXXXXX", bgp_peer_state="available", bgp_status="up", exported_job="awd-dc", instance="localhost:9980", job="aws-dc", service="direct-connect"}
sergey-safarov commented 3 years ago

Sorry, I didn't get you. You could make queries with aws_dc_connections{} / aws_dc_bgp_peers{} on PromQL. If you wanna see all the labels that are exported, just make this empty query on Prometheus, it'll show a list of labels.

Hello @mr-karan Vlad asking about AWS Connection and Virtual Interface tags export from AWS cloud into aws-dc-exporter /metrics output.

Our use case. Via AWS console we tag Connections and Virtual Interface with the tag "notification_type". This tag may have values:

  1. prod
  2. stage
  3. dev
  4. maintenance

We expect to get this tag in exporter output like

aws_dc_connections{tag_notification_type="maintenance",job="myjob",conn_state="available",conn_name="EVYX/279726//ZYO Centracom Inc",conn_id="dxcon-fhbwcw0d",bandwidth="50Mbps"} 1

using this tag_notification_type value we create Prometheus rule to raise alarm. If connections have tag_notification_type=maintenance then do not rize alarm.

Could you suggest how can be implemented Connections and Virtual Interface tags exporting? I and Vlad not Go devs and simple do know how to dynamically add keys to output.

mr-karan commented 3 years ago

Ah that makes sense. We'll need to use https://docs.aws.amazon.com/sdk-for-go/api/service/directconnect/#DirectConnect.DescribeTags to collect these tags.

I'll add this to my TODO but a bit busy these days so will try to ship it this week. Thanks!