Airflow Variable Events (Create, Edit, Delete) Do Not Appear In The Audit Log If Triggered From A DAG File

[karenbraganz]

Apache Airflow version

2.9.3

If "Other Airflow 2 version" selected, which one?

No response

What happened?

If an Airflow variable is created, updated, or deleted from a DAG file using Variable.set(), Variable.update(), or Variable.delete() methods, these events do not appear in the audit log. On the other hand, if a variable is created, updated, or deleted directly in the UI, these events appear in the audit logs.

What you think should happen instead?

Create, update, and delete events for Airflow variables should appear in audit logs even if they are triggered from a DAG file. After reviewing this PR, I think this can be fixed by adding the @action_logging decorator to the set(), update(), and delete() methods in airflow.models.Variable. I'm not sure if this is the correct solution or if other changes will also need to be made.

How to reproduce

1. Create a DAG containing a task with one or all of these methods:

   • Variable.set()
   • Variable.update()
   • Variable.delete()

2. Trigger a DAG run and let it complete.

3. Check the cluster audit logs for events relating to variable creation, updates, or deletion. These only appear if the changes are made directly on the UI.

Operating System

MacOS

Versions of Apache Airflow Providers

No response

Deployment

Astronomer

Deployment details

Tested on Astro CLI

Anything else?

No response

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[boring-cyborg[bot]]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[pushpendu91]

Even though we can add below code for example in the set method to add audit log for variable creation from DAG and it is working fine as shown in the screenshot, but without proper dag_id, owner etc. it does not makes sense.

session.add( Log( event="variable.create", dag_id="", owner="scheduler", owner_display_name="Scheduler", extra=f"key={key}, val={stored_value}, description={description}", ) )

And because airflow.models.variable does not have dag or dagrun context it is not possible without some core code changes to get the dag_id, task_id, run_id, owner etc. details to be logged. If community agrees we can add these basic details(event, extra, owner=‘scheduler’ etc.) for set, update and delete method for Variables.

[nicolasge]

Personally I believe it's a great feature we need to have, otherwise we don't know who changed the variables within the DAGs.