search

epoch8 / airflow-exporter

Airflow plugin to export dag and task based metrics to Prometheus.
Other
240 stars 75 forks source link

DAG not found in serialized_dag table #98

Closed nvn01234 closed 2 years ago

nvn01234 commented 3 years ago

Step by step to reproduce the bug:

Currently I have to run a CLI command: airflow dags delete <dag_id> to completely remove all records related to a deleted dag

The solution can be: Ignore the deleted dags (don't show them in metrics)

/label ~bug

elephantum commented 3 years ago

Interesting, thanks for reporting.

sawaca96 commented 3 years ago

@nvn01234 I experienced the same thing.

In my case, the error occurred if it disappeared from UI due to git-sync but remained in the DB

you solve this error just delete dag from DB. Use airflow cli airflow dags delete <dag_id> -y

biryukovsky commented 3 years ago

I had a similar problem. In my case the exception message was like "airflow.exception.SerializedDagNoFound: DAG 'None' not found in serialized_dag table". Fixed with following query in airflow database: DELETE FROM dag_run WHERE dag_id IS NULL;

elephantum commented 3 years ago

Hi, everyone!

If anyone would have time to create a PR that fixes it, I would be grateful. Fix should not be too complex.

sawaca96 commented 3 years ago

@elephantum How should I fix it? just fix it to ignore airflow.exceptions.SerializedDagNotFound ?

elephantum commented 3 years ago

I would suggest altering sqlalchemy query for this case, most probably simple join with dag table would be enough.

I.e. filter out such tasks and dags that do not have corresponding dag in DB.

sawaca96 commented 2 years ago

@elephantum #105 I create PR Thank you for the opportunity.😁😁

CyberHippo commented 2 years ago

Experienced it as well.

baryluk commented 2 years ago

Same issue here, with airflow-exporter 1.5.1,

and airflow 1.10.15, (where for some very old dags we do not have serialized_dag entries, as we only enabled that option few weeks ago in preparation for the Airflow 2.x update), but it will be an issue when I upgrade to 2.x, because we will be preserving full history.

I have a single dag in dag table, that is no longer available. Looking at postgresql database it has is_paused false, is_subdag false, is_active false.

elephantum commented 2 years ago

Fixed in v1.5.2

baryluk commented 2 years ago

Excellent, it works. Thanks @sawaca96 and @elephantum