airflow exporter is failing on /admin/metrics page

[pindge]

Ooops!
Something bad has happened.

Airflow is used by many users, and it is very likely that others had similar problems and you can easily find
a solution to your problem.

Consider following these steps:

  * gather the relevant information (detailed logs with errors, reproduction steps, details of your deployment)

  * find similar issues using:
     * GitHub Discussions
     * GitHub Issues
     * Stack Overflow
     * the usual search engine you use on a daily basis

  * if you run Airflow on a Managed Service, consider opening an issue using the service support channels

  * if you tried and have difficulty with diagnosing and fixing the problem yourself, consider creating a bug report.
    Make sure however, to include all relevant details and results of your investigation so far.

Python version: 3.8.12
Airflow version: 2.2.2
Node: airflow-web-7f59787854-87msz
-------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/airflow/.local/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow_exporter/prometheus_exporter.py", line 326, in list
    return Response(generate_latest(), mimetype='text')
  File "/home/airflow/.local/lib/python3.8/site-packages/prometheus_client/exposition.py", line 176, in generate_latest
    for metric in registry.collect():
  File "/home/airflow/.local/lib/python3.8/site-packages/prometheus_client/registry.py", line 83, in collect
    for metric in collector.collect():
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow_exporter/prometheus_exporter.py", line 233, in collect
    labels = get_dag_labels(dag.dag_id)
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow_exporter/prometheus_exporter.py", line 201, in get_dag_labels
    labels = labels.get('__var', {})
AttributeError: 'Param' object has no attribute 'get'

[elephantum]

@pindge can you describe your setup?

Do you have a way to reproduce this behavior?

[pindge]

@pindge can you describe your setup?

Do you have a way to reproduce this behavior?

git clone https://github.com/GeoscienceAustralia/dea-airflow.git
cd dea-airflow
git checkout develop

mkdir ./logs
echo -e "AIRFLOW_UID=$(id -u)\nAIRFLOW_GID=0" >> .env
docker-compose up airflow-init
docker-compose up

via web browser

1. username: airflow and password:airflow
2. enable dag dea_public_data_sns_streamline_indexing
3. navigate to /admin/metrics/ end point
4. then there is the error

Likely due to this line

    params={"labels": {"env": "dev"}},

here is the snippets, I am going to fix this, but will leave this issue open

# THE DAG
dag = DAG(
    "dea_public_data_sns_streamline_indexing",
    doc_md=__doc__,
    default_args=DEFAULT_ARGS,
    schedule_interval="0 */1 * * *",  # hourly
    # schedule_interval=None,  # for testing
    catchup=False,
    max_active_runs=1,
    tags=["k8s", "sentinel-2", "streamline-indexing"],
    params={"labels": {"env": "dev"}},
)

[tanvn]

I got the same error the day before on Airflow 2.2.2 too. I guessed from 2.2.2, the Param class has been changed https://github.com/apache/airflow/blob/main/airflow/models/param.py#L36 so we need to change the way to get the labels on

  File "/home/airflow/.local/lib/python3.8/site-packages/airflow_exporter/prometheus_exporter.py", line 201, in get_dag_labels
    labels = labels.get('__var', {})

This function (get_dag_labels) runs well on Airflow 2.1.1

[elephantum]

I'm looking into it