DAG Import Errors not show in webUI if DAG is first time upload to airflow is not currect

sunny760408 commented 2 years ago

Apache Airflow version

2.2.2

What happened

Hi everyone,

In order to separate permission, I create two roles:

CustomRole: Copy from User, and then delete "an read on DAGs, can read on DAG Runs" Permissions is as below:

[can read on DAG Runs, can read on Task Instances, can edit on Task Instances, can delete on DAG Runs, 
can create on DAG Runs, can read on Audit Logs, **can read on ImportError,** can read on XComs, can read on DAG Code, 
can read on Plugins, can read on DAG Dependencies, can read on Jobs, can read on My Password, can edit on My Password, 
can read on My Profile, can edit on My Profile, can read on SLA Misses, can read on Task Logs, 
can read on Website, menu access on Browse, menu access on DAG Runs, menu access on Documentation, 
menu access on Docs, menu access on Jobs, menu access on Audit Logs, menu access on Plugins, menu access on SLA Misses, 
menu access on Task Instances, can delete on DAGs, can create on Task Instances, can delete on Task Instances, 
can edit on DAG Runs]

TeamA: Default have "can read on Website", and each DAG upload to airflow will add can read on DAG and can edit on DAG Permissions is as below:
```
[can read on Website]
```

TeamA_DAG_apple.py access control:

...
with DAG(
    dag_id="TeamA_DAG_apple",
    schedule_interval=None,
    start_date = days_ago(1),
    catchup = False,
    tags=["TeamA"],
    access_control={
        'TeamA': {'can_read', 'can_edit'}
    }
....

Now I upload a DAG name TeamA_DAG_apple, the permissions will change as below:

[can read on Website, can read on DAG:TeamA_DAG_apple, can edit on DAG:TeamA_DAG_apple]

Create a user have two roles [CustomRole,TeamA]
Here is the issue:
- If first time upload DAG "TeamA_DAG_apple" is correct , the DAG can be added to permission in role "TeamA", and then you modify schedule_interval with wrong format, such as schedule_interval="10 30 *", you can see DAG Import Error show in WebUI.
- But If first time upload DAG "TeamA_DAG_apple" is NOT correct you can't see DAG Import Error show in WebUI, because DAG not be added to permission in role "TeamA", an error DAG not able to be added to permission.
- Only user who have permission [can read on DAGs] can see DAG Import Error.

I guess airflow check DAG first not add permission first, so change the order maybe can solve this issue. Because I upload an new incorrect DAG, I can see DAG Import Errors by login as Admin, but I can't add "can read on DAG:TeamA_DAG_apple, can edit on DAG:TeamA_DAG_apple" permission to role TeamA.

What you expected to happen

No response

How to reproduce

Step 1. Create a role name CustomRole with follow permission. [can read on DAG Runs, can read on Task Instances, can edit on Task Instances, can delete on DAG Runs, can create on DAG Runs, can read on Audit Logs, can read on ImportError, can read on XComs, can read on DAG Code, can read on Plugins, can read on DAG Dependencies, can read on Jobs, can read on My Password, can edit on My Password, can read on My Profile, can edit on My Profile, can read on SLA Misses, can read on Task Logs, can read on Website, menu access on Browse, menu access on DAG Runs, menu access on Documentation, menu access on Docs, menu access on Jobs, menu access on Audit Logs, menu access on Plugins, menu access on SLA Misses, menu access on Task Instances, can delete on DAGs, can create on Task Instances, can delete on Task Instances, can edit on DAG Runs]

Step 2. Create a role name TeamA with follow permission. [can read on Website]

Step 3. Create a user with role CustomerRole and TeamA.

Step 4. Upload an incorrect DAG name TeamA_DAG_banana. I'd like to use wrong schedule_interval="1 30 *".

Step 5. Add permission [can read on DAG:TeamA_DAG_banana, can edit on DAG:TeamA_DAG_banana] to role TeamA. But can not add. Also can not see DAG import Error.

Step 6. Change login user to Admin, and can see DAG import Error.

Operating System

PRETTY_NAME="Debian GNU/Linux 10 (buster)" NAME="Debian GNU/Linux" VERSION_ID="10" VERSION="10 (buster)" VERSION_CODENAME=buster ID=debian

Versions of Apache Airflow Providers

apache-airflow 2.2.2 apache-airflow-providers-amazon 2.4.0 apache-airflow-providers-celery 2.1.0 apache-airflow-providers-cncf-kubernetes 2.1.0 apache-airflow-providers-docker 2.3.0 apache-airflow-providers-elasticsearch 2.1.0 apache-airflow-providers-ftp 2.0.1 apache-airflow-providers-google 6.1.0 apache-airflow-providers-grpc 2.0.1 apache-airflow-providers-hashicorp 2.1.1 apache-airflow-providers-http 2.0.1 apache-airflow-providers-imap 2.0.1 apache-airflow-providers-microsoft-azure 3.3.0 apache-airflow-providers-microsoft-mssql 2.0.1 apache-airflow-providers-mysql 2.1.1 apache-airflow-providers-odbc 2.0.1 apache-airflow-providers-oracle 2.0.0 apache-airflow-providers-postgres 2.3.0 apache-airflow-providers-redis 2.0.1 apache-airflow-providers-sendgrid 2.0.1 apache-airflow-providers-sftp 2.2.0 apache-airflow-providers-slack 4.1.0 apache-airflow-providers-sqlite 2.0.1 apache-airflow-providers-ssh 2.3.0

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

potiuk commented 2 years ago

Feel free to take a look at it.

Venture200 commented 2 years ago

Hello, I have a similar issue that with Admin role I can see Import Errors but with Regular role I can't.

ZdravkoHvarlingovFt commented 2 years ago

Hey, I can confirm the problem exists as well. 😢 That was not the case with version 1 of Airflow. I can see in the changelog that some changes were introduced in version 2 in order to prevent a user to see import errors of other teams' DAGs. Probably that fix introduced that side effect we are talking about here.

bmoon4 commented 1 year ago

Hi, im seeing same issue in airflow 2.5.1.. If a new DAG has import errors, error message does not appear in GUI. Only Admin role users can see it but other role users with 'read on import errors' permission can't.

github-actions[bot] commented 7 months ago

This issue has been automatically marked as stale because it has been open for 365 days without any activity. There has been several Airflow releases since last activity on this issue. Kindly asking to recheck the report against latest Airflow version and let us know if the issue is reproducible. The issue will be closed in next 30 days if no further activity occurs from the issue author.

apache / airflow