Warning about invisible connections/variable in the Web UI.

apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

https://airflow.apache.org/

Apache License 2.0

37.03k stars 14.28k forks source link

Warning about invisible connections/variable in the Web UI. #10867

Open mik-laj opened 4 years ago

mik-laj commented 4 years ago

Hello,

Currently, you can define a new connection/variable in several ways.

Metabase
Environment variable
Secret backend

However, only the connections defined with the Metabase are visible from the Web UI. This is surprising among users because a natural way to check connection/variable configuration is to use the Web UI, which may not show complete information. I would be happy if there was a warning stating that not all connections / variables are visible when the user has a configured secret backend or when the AIRFLOW__CONN_* /AIRFLOW__VAR_* environment variable is defined. This should appear above the list of connections/variables.

This change is intended to improve the user experience.

Best regards, Kamil Breguła

JeffryMAC commented 4 years ago

If a connection defined in one of those hidden options. What will happen if I will try to define a new connection in the UI with the exact name?

randr97 commented 4 years ago

If a connection defined in one of those hidden options. What will happen if I will try to define a new connection in the UI with the exact name?

It will override the env. (currently) But based on suggested changes, we should jus warn the user and nothing more. @mik-laj Should we consider having the env conn/vars displayed in UI or jus show a warning?

mik-laj commented 4 years ago

@randr97 For security reasons, we should only display database entries in the Web UI. The entries that are configured in the database are user-controlled entries and the user can fully manage them.

@JeffryMAC will be created but never used, because backend entries with higher priority will be used. https://github.com/apache/airflow/blob/c58d60635dbab1a91f38e989f72f91645cb7eb62/airflow/configuration.py#L985-L1001 https://github.com/apache/airflow/blob/c58d60635dbab1a91f38e989f72f91645cb7eb62/airflow/secrets/__init__.py#L29-L32 https://github.com/apache/airflow/blob/c58d60635dbab1a91f38e989f72f91645cb7eb62/airflow/models/connection.py#L386-L397

kaxil commented 4 years ago

Yes we should definitely not display Connections and Variable coming from Secrets Backend, I am happy with us showing just the keys for debugging but not values

potiuk commented 4 years ago

+1. It would be nice to prevent creating one if there is a secret backend one already but it's not really necessary

kaxil commented 2 years ago

@blag will take care of it for 2.3.0 if no-one does it by then