search

nerdswords / yet-another-cloudwatch-exporter

Prometheus exporter for AWS CloudWatch - Discovers services through AWS tags, gets CloudWatch metrics data and provides them as Prometheus metrics with AWS tags as labels
Apache License 2.0
972 stars 332 forks source link

[BUG] Missing NLB metrics after upgrade to 0.38-alpha #639

Open 12ushan opened 2 years ago

12ushan commented 2 years ago

Is there an existing issue for this?

Current Behavior

Upgraded yet-another-cloudwatch-exporter from 0.32 to 0.38 and found NLB metrics that were previously being scraped missing. No changes to the NLB resources itself and no indication of errors in YACE logs.

Attached prometheus screengrab shows no NLB metrics being returned after upgrading YACE on 8th Augus

Screenshot 2022-08-15 at 11 10 07

t

Expected Behavior

NLB metrics being scraped as before.

Steps To Reproduce

YACE version 0.38.0-alpha running in kubernetes cluster. Config for NLB which has stayed the same throughout the upgrade:

Anything else?

No response

nhinds commented 1 year ago

I'm running into something similar. aws_*_info metrics for any services are not being returned if those jobs don't have any metrics. For me this is common for NLB metrics, because NLB metrics are absent when there's no data. In your case, if there are no unhealthy hosts in your NLB then there will be no datapoints for UnHealthyHostCount, and no aws_nlb_info metrics will be returned.

If any resource in your job/region returns at least 1 metric, the aws_*_info metrics reappear.

This seems to have been a side-effect of #606, where resources and metrics are only recorded if there's at least one resource and at least one metric.