Closed YonatanKiron closed 8 months ago
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
Another way to reproduce:
from airflow.stats import Stats
Stats.timing(f"dagrun.duration.success", **{"dt":1000, "tags":{"dag_id": "bla"}})
Stats.timing(f"dagrun.duration.success", **{"dt":1000, "tags":{"dag_id": "bla2"}})
Stats.instance.metrics_map.map
#### {'airflow.dagrun.duration.success_dag_id_bla': Observation(value=1000.0, attributes={'dag_id': 'bla'}),
#### 'airflow.dagrun.duration.success_dag_id_bla2': Observation(value=1000.0, attributes={'dag_id': 'bla2'})}
The debug log will report:
{
"resource_metrics": [
{
"resource": {
"attributes": {
"service.name": "Airflow"
},
"schema_url": ""
},
"scope_metrics": [
{
"scope": {
"name": "airflow.metrics.otel_logger",
"version": null,
"schema_url": ""
},
"metrics": [
{
"name": "airflow.dagrun.duration.success",
"description": "",
"unit": "",
"data": {
"data_points": [
{
"attributes": {
"dag_id": "bla"
},
"start_time_unix_nano": 0,
"time_unix_nano": 1697698348843414114,
"value": 1000.0
}
]
}
}
],
"schema_url": ""
}
],
"schema_url": ""
}
]
}
So my assumption, is that the issue is value override was wrong and the issue is somewhere on the reporting part
Hey, since this is an Otel issue and not an Airflow issue I want to close the ticket. Feel free to reopen if you disagree.
Apache Airflow version
2.7.2
What happened
I have two DAGs that constantly pass, one reported to otel agent while the other didn't.
After looking into the logs (in debug mode) it seems like the metric
airflow.dagrun.duration.success
contains only one value.What you think should happen instead
airflow.dagrun.duration.success value should contain a list of values instead of one
How to reproduce
Create 2 dummy DAGs that pass and watch the metrics
Operating System
Docker bitnami/airflow:2.7.2
Versions of Apache Airflow Providers
Deployment
Other 3rd-party Helm chart
Deployment details
bitnami/airflow
Anything else
I suspect that this code block is overriding the metrics
key
every time instead of appending it's values in case of existencehttps://github.com/apache/airflow/blob/2.7.2/airflow/metrics/otel_logger.py#L327-L343
Are you willing to submit PR?
Code of Conduct