Get all metrics without selecting by dimension

[jangrewe]

I thought i'd open a separate issue to not hijack my previous ticket, sooo...

I haven't actually double-checked, but i think with the "official" cloudwatch-exporter it's possible to omit aws_dimension_select and just get all metrics for a specific dimension. The config for this would look like the following:

---
tasks:
- name: vpn
  metrics:
  - aws_namespace: "AWS/VPN"
    aws_metric_name: TunnelState
    aws_dimensions: [VpnId]
    aws_statistics: [Average]

Unfortunately this crashes the exporter:

CloudWatch exporter started...
InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, GetMetricStatisticsInput.Dimensions[0].Value.

panic: inconsistent label cardinality

goroutine 349 [running]:
github.com/Technofy/cloudwatch_exporter/vendor/github.com/prometheus/client_golang/prometheus.MustNewConstMetric(0xc420139490, 0x2, 0x3fe0000000000000, 0xc420316920, 0x1, 0x2, 0x0, 0x1)
    /home/jan/go/src/github.com/Technofy/cloudwatch_exporter/vendor/github.com/prometheus/client_golang/prometheus/value.go:172 +0xb0
main.scrape(0xc420326c80, 0xc4205689c0)
    /home/jan/go/src/github.com/Technofy/cloudwatch_exporter/aws.go:100 +0x1047
main.(*cwCollector).Collect(0xc420326c80, 0xc4205689c0)
    /home/jan/go/src/github.com/Technofy/cloudwatch_exporter/collector.go:111 +0x59
github.com/Technofy/cloudwatch_exporter/vendor/github.com/prometheus/client_golang/prometheus.(*Registry).Gather.func2(0xc420313840, 0xc4205689c0, 0x9dc1e0, 0xc420326c80)
    /home/jan/go/src/github.com/Technofy/cloudwatch_exporter/vendor/github.com/prometheus/client_golang/prometheus/registry.go:433 +0x61
created by github.com/Technofy/cloudwatch_exporter/vendor/github.com/prometheus/client_golang/prometheus.(*Registry).Gather
    /home/jan/go/src/github.com/Technofy/cloudwatch_exporter/vendor/github.com/prometheus/client_golang/prometheus/registry.go:434 +0x2ec

[odupuy]

@jangrewe

I have no VPN to test but with the changes included in #4 then #6 with the support of regexp and default this one which lloks similar works perfectly.

 - name: test-ec2
   metrics:
    - aws_namespace: "AWS/EC2"
      aws_metric_name: CPUUtilization
      aws_dimensions: [ImageId]
      aws_statistics: [Average]

curl -sS "http://localhost:9042/scrape?task=test-ec2&region=us-east-1" giving

# H# HELP aws_ec2_cpu_utilization_average CPUUtilization
# TYPE aws_ec2_cpu_utilization_average gauge
aws_ec2_cpu_utilization_average{image_id="ami-3xxxxxxx",task="test-ec2"} 1.8866666666666667
aws_ec2_cpu_utilization_average{image_id="ami-9xxxxxxx",task="test-ec2"} 15.225238095238183
aws_ec2_cpu_utilization_average{image_id="ami-axxxxxxx",task="test-ec2"} 2.8630000000000004
aws_ec2_cpu_utilization_average{image_id="i-xxxxxxxxxxxxx",task="test-ec2"} 23.67
aws_ec2_cpu_utilization_average{image_id="i-xxxxxxxxxxxxx",task="test-ec2"} 1.75
...
aws_ec2_cpu_utilization_average{image_id="i-xxxxxxxxxxxxx",task="test-ec2"} 1.83
aws_ec2_cpu_utilization_average{image_id="i-xxxxxxxxxxxxx",task="test-ec2"} 1.8
aws_ec2_cpu_utilization_average{image_id="m3.medium",task="test-ec2"} 8.83
aws_ec2_cpu_utilization_average{image_id="m4.large",task="test-ec2"} 1.858
aws_ec2_cpu_utilization_average{image_id="t2.nano",task="test-ec2"} 0.166666666667273
# HELP cloudwatch_exporter_erroneous_requests The number of erroneous request made by this scrape.
# TYPE cloudwatch_exporter_erroneous_requests gauge
cloudwatch_exporter_erroneous_requests 0
# HELP cloudwatch_exporter_scrape_duration_seconds Time this CloudWatch scrape took, in seconds.
# TYPE cloudwatch_exporter_scrape_duration_seconds gauge
cloudwatch_exporter_scrape_duration_seconds 24.668273297

[jhinklew]

I'm getting the same error @jangrewe got but with an RDS task

curl localhost:9042/scrape?task=rds

giving

panic: inconsistent label cardinality

goroutine 308 [running]:
github.com/prometheus/client_golang/prometheus.MustNewConstMetric(0xc42017dce0, 0x2, 0x403ca147ae147ae1, 0xc4200d8c60, 0x1, 0x2, 0x0, 0x1)
    /Users/anthony.teisseire/go/src/github.com/prometheus/client_golang/prometheus/value.go:174 +0xb3
main.scrape(0xc4202860f0, 0xc420052360)
    /Users/anthony.teisseire/go/src/github.com/Technofy/cloudwatch_exporter/aws.go:104 +0xf4a
main.(*cwCollector).Collect(0xc4202860f0, 0xc420052360)
    /Users/anthony.teisseire/go/src/github.com/Technofy/cloudwatch_exporter/collector.go:111 +0x5b
github.com/prometheus/client_golang/prometheus.(*Registry).Gather.func2(0xc42001a4c0, 0xc420052360, 0xa55700, 0xc4202860f0)
    /Users/anthony.teisseire/go/src/github.com/prometheus/client_golang/prometheus/registry.go:383 +0x61
created by github.com/prometheus/client_golang/prometheus.(*Registry).Gather
    /Users/anthony.teisseire/go/src/github.com/prometheus/client_golang/prometheus/registry.go:381 +0x2e1

Tasks config

  tasks:
  - name: rds
    default_region: us-east-1
    metrics:
      - aws_namespace: AWS/RDS
        aws_metric_name: CPUUtilization
        aws_dimensions: [DBInstanceIdentifier]
        aws_statistics: [Maximum]

[a-teisseire]

Hi @jhinklew the issue you're facing has been resolved in the code IIRC, but we didn't update the Docker image on the Hub. I will try to update it but I don't have much time to maintain these projects. In the meantime, if you want to try if the exporter will do the job for you, you can clone the code and build it. Please do not hesitate to post your results!

[mvitale1989]

I can confirm that with the latest version of the code, i can freely omit both aws_dimension_select and aws_dimension_select_regex without the exporter crashing. Thank you!