search

epoch8 / airflow-exporter

Airflow plugin to export dag and task based metrics to Prometheus.
Other
240 stars 75 forks source link

Custom labels not working after upgrade to Airflow 2.3.1 #113

Open akepa opened 2 years ago

akepa commented 2 years ago

Airflow version: 2.3.1 airflow-exporter: 1.5.3

Hello,

we have upgraded recently our Airflow from 2.2.5 to 2.3.1 and we have seen that after the upgrade, the feature to include custom labels in the metrics is not working anymore.

with DAG(
        dag_id='my_dag',
        schedule_interval="10 * * * *",
        start_date=days_ago(1),
        default_args={},
        max_active_runs=1,
        tags=['tag1', 'tag2'],
        params={'labels': {
            'custom_label': 'custom_label_value'
        }}
) as dag:
...

with previous version, /metrics endpoint was exposing the following metrics:

airflow_dag_status{dag_id="my_dag",owner="airflow",status="failed", custom_label="custom_label_value"} 169.0
airflow_dag_status{dag_id="my_dag",owner="airflow",status="running", custom_label="custom_label_value"} 1.0
airflow_dag_status{dag_id="my_dag",owner="airflow",status="success", custom_label="custom_label_value"} 203.0

after the upgrade, it is not including the custom label

airflow_dag_status{dag_id="my_dag",owner="airflow",status="failed"} 169.0
airflow_dag_status{dag_id="my_dag",owner="airflow",status="running"} 1.0
airflow_dag_status{dag_id="my_dag",owner="airflow",status="success"} 203.0
akepa commented 2 years ago

Seems to be related to https://github.com/epoch8/airflow-exporter/blob/master/airflow_exporter/prometheus_exporter.py#L203.

I've been debugging in local with 2.3.2 and 

dag.params.get('labels', {})

has now the following content:

{'custom_label': 'custom_label_value'}