apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
https://airflow.apache.org/
Apache License 2.0
36.53k stars 14.15k forks source link

Parsing error of email_conn_id on utils/email.py #23262

Open nivangio opened 2 years ago

nivangio commented 2 years ago

Apache Airflow version

2.2.2

What happened

When picking email_conn_id from Secrets Manager, the connection is correctly retrieved from ASM but parsed as an URI. For that reason, login and password parameters are left empty and therefore not picked in send_mime_email (lines 189,190).

The connection is stored as follows:

{"login": "my_email@adress.com", "password": "my_password"}

What you think should happen instead

No response

How to reproduce

No response

Operating System

Ubuntu 20.04

Versions of Apache Airflow Providers

No response

Deployment

Docker-Compose

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

Code of Conduct

boring-cyborg[bot] commented 2 years ago

Thanks for opening your first issue here! Be sure to follow the issue template!

mik-laj commented 2 years ago

The connection is stored as follows:

{"login": "my_email@adress.com", "password": "my_password"}

According to the documentation, the connection should be written as a URI.

You can store the different values for a secret in two forms: storing the conn URI in one field (default mode) or using different fields in Amazon Secrets Manager (setting full_url_mode as false in the backend config), as follows:

https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/secrets-backends/aws-secrets-manager.html

By default, we use URI formaat, so you should use the following command to set URI:

aws secretsmanager put-secret-value \
    --secret-id airflow/connections/smtp_default \
    --secret-string "smtps://user:host@relay.example.com:465"

https://airflow.apache.org/docs/apache-airflow-providers-amazon/2.2.0/secrets-backends/aws-secrets-manager.html

nivangio commented 2 years ago

I finally managed to do it. It required for me to set conn_type, port and host in the connection secret in AWS (I'm working with full_url_mode: false) as well as placing them in the config because if the URI is not complete, login and password are not correctly parsed

It'd be nice to have just one set-up instead. Thx!

potiuk commented 2 years ago

Yeah. PRs to fix it are most welcome @nivangio !

dbqinhust commented 1 year ago

Could you assign it to me?