apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
https://airflow.apache.org/
Apache License 2.0
37.48k stars 14.37k forks source link

Variable <name> does not exist when setting as environment variable #30122

Closed ryandutton closed 1 year ago

ryandutton commented 1 year ago

Apache Airflow version

Other Airflow 2 version (please specify below)

What happened

In Airflow 2.3.3, when setting an environment variable such as AIRFLOW_VAR_ENVIRONMENT to testing I am getting a Broken Dag error. The error message is

Broken DAG: [/blah/file.py] Traceback (most recent call last):
  File "/blah/environment.py", line 62, in _get_environment
    return Variable.get('environment')
  File "/usr/local/autotrader/lib/python3.8/site-packages/airflow/models/variable.py", line 138, in get
    raise KeyError(f'Variable {key} does not exist')
KeyError: 'Variable environment does not exist'

I've checked the database and the variable is not set in there, which may be the expected behaviour as it doesn't show in the UI which is expected. When I run airflow variables get environment it does return the expected value, however, in a dag running `Variables.get('environment') it doesn't get the variable.

What you think should happen instead

The expected behaviour would be that when running Variables.get('environment') in a dag, that it would return the expected value.

How to reproduce

Set a variable as an environment variable and try getting the variable in a dag using `Variables.get(''). This would result in a Broken Dag error.

Operating System

Rocky Linux 8.7

Versions of Apache Airflow Providers

Airflow v2.3.3

Deployment

Other 3rd-party Helm chart

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

Code of Conduct

boring-cyborg[bot] commented 1 year ago

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.