[BUG] VPN metrics missing dimensions

igor-nikiforov commented 10 months ago

Is there an existing issue for this?

[X] I have searched the existing issues

YACE version

v0.51.0

Config file

  - type: AWS/VPN
    regions:
      - us-east-1
    period: 300
    length: 300
    metrics:
      - name: TunnelDataIn
        statistics: [Sum]
      - name: TunnelDataOut
        statistics: [Sum]
      - name: TunnelState
        statistics: [Maximum]

Current Behavior

Below config gives me following output:

# HELP aws_vpn_tunnel_state_maximum Help is not implemented yet.
# TYPE aws_vpn_tunnel_state_maximum gauge
aws_vpn_tunnel_state_maximum{account_id="xxx",dimension_TunnelIpAddress="",dimension_VpnId="",name="global",region="us-east-1"} 1
aws_vpn_tunnel_state_maximum{account_id="xxx",dimension_TunnelIpAddress="",dimension_VpnId="vpn-030c3b1ac3519b521",name="arn:aws:ec2:us-east-1:xxx:vpn-connection/vpn-030c3b1ac3519b521",region="us-east-1"} 1
aws_vpn_tunnel_state_maximum{account_id="xxx",dimension_TunnelIpAddress="",dimension_VpnId="vpn-05c2b8f94e967c0c6",name="arn:aws:ec2:us-east-1:xxx:vpn-connection/vpn-05c2b8f94e967c0c6",region="us-east-1"} 1
aws_vpn_tunnel_state_maximum{account_id="xxx",dimension_TunnelIpAddress="",dimension_VpnId="vpn-8bbfa1ea",name="arn:aws:ec2:us-east-1:xxx:vpn-connection/vpn-8bbfa1ea",region="us-east-1"} 1
aws_vpn_tunnel_state_maximum{account_id="xxx",dimension_TunnelIpAddress="3.xxx.xxx.81",dimension_VpnId="",name="global",region="us-east-1"} 1
aws_vpn_tunnel_state_maximum{account_id="xxx",dimension_TunnelIpAddress="34.xxx.xxx.115",dimension_VpnId="",name="global",region="us-east-1"} 1
aws_vpn_tunnel_state_maximum{account_id="xxx",dimension_TunnelIpAddress="34.xxx.xxx.29",dimension_VpnId="",name="global",region="us-east-1"} 1
aws_vpn_tunnel_state_maximum{account_id="xxx",dimension_TunnelIpAddress="34.xxx.xxx.212",dimension_VpnId="",name="global",region="us-east-1"} 1
aws_vpn_tunnel_state_maximum{account_id="xxx",dimension_TunnelIpAddress="34.xxx.xxx.255",dimension_VpnId="",name="global",region="us-east-1"} 1
aws_vpn_tunnel_state_maximum{account_id="xxx",dimension_TunnelIpAddress="52.xxx.xxx.211",dimension_VpnId="",name="global",region="us-east-1"} 0

As you see here there is no relation between TunnelIpAddress and VpnId. As a result you can't understand which pair of tunnels relates to which VPN ids.

If you try to add this to config:

    dimensionNameRequirements:
      - VpnId

Metrics will be squashed without info about TunnelIpAddress:

# TYPE aws_vpn_tunnel_state_maximum gauge
aws_vpn_tunnel_state_maximum{account_id="xxx",dimension_VpnId="vpn-030c3b1ac3519b521",name="arn:aws:ec2:us-east-1:xxx:vpn-connection/vpn-030c3b1ac3519b521",region="us-east-1"} 1
aws_vpn_tunnel_state_maximum{account_id="xxx",dimension_VpnId="vpn-05c2b8f94e967c0c6",name="arn:aws:ec2:us-east-1:xxx:vpn-connection/vpn-05c2b8f94e967c0c6",region="us-east-1"} 1
aws_vpn_tunnel_state_maximum{account_id="xxx",dimension_VpnId="vpn-8bbfa1ea",name="arn:aws:ec2:us-east-1:xxx:vpn-connection/vpn-8bbfa1ea",region="us-east-1"} 1

Expected Behavior

TunnelIpAddress and VpnId should appear both in all VPN metrics.

Steps To Reproduce

No response

Anything else?

No response

alibner commented 3 months ago

yeah, the same issue about AWS/Lambda. But referring to https://github.com/nerdswords/yet-another-cloudwatch-exporter/issues/1272, you should probably drop the value of the metric name as a global metric.

kgeckhart commented 3 months ago

@igor-nikiforov what you are seeing is what CloudWatch provides OOTB for those metrics. The ARN for a VPN connection only contains the VPN ID, which is the only thing we have available for matching dimensions https://github.com/nerdswords/yet-another-cloudwatch-exporter/blob/master/pkg/config/services.go#L811-L820

This results in metrics which only have the TunnelIpAddress dimension with a resource.

Given the current setup of the exporter this is working as intended and your request is for a new feature. I'm guessing there is another API available which could be used to figure out the relationship between TunnelIpAddress and VpnId. Being able to expand the scope of resource metadata + relationship data is something we are looking to support to help with this extra valuable enrichment.

nerdswords / yet-another-cloudwatch-exporter