How to save sasl.password in a masked field in kafka connection?

[151N3]

We have a new project, where we have to use sasl-Username and sasl-Password and don't want put the password in the extra-Dict. Is there any way to save the credentials somewhere, where the password is not in plain text?

I'm missing a password or a masked filed, where we able to store a dictionary with sensible data and maybe in the hook merge both dictionaries from extra and the other field. Something like this:

extra = {
    "bootstrap.servers": "kafka:19092",
    "group.id": "group_1",
    "security.protocol": "PLAINTEXT",
    "auto.offset.reset": "beginning"
}
new_field = {
   "sasl.username":"my_user",
   "sasl.password": "my_password"}
config_dict = extra | new_field

[dylanbstorey]

Variables / Secrets should be an ok work around - your other option would be to put the extra information in the actual connection as its not stored in plain text there.

You should also begin using https://airflow.apache.org/docs/apache-airflow-providers-apache-kafka/stable/index.html as a stable supported plugin.

[151N3]

Hello @dylanbstorey , thank you for your reply. I'm not sure if I understood your answer. Regarding to the Provider, I'm using: apache-airflow==2.9.0 apache-airflow-providers-apache-kafka==1.3.1

I did a local Upgrade to airflow 2.9.2 and apache-kafka=1.4.1 but I don't see an changes at the connection level.

I'm not using any other Secrets Backend and I have to rely on the Connections and Variables.

Can you please explain to me what do you mean by this: your other option would be to put the extra information in the actual connection as its not stored in plain text there. ?

[dylanbstorey]

https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/variables.html

extra = {
    "bootstrap.servers": "kafka:19092",
    "group.id": "group_1",
    "security.protocol": "PLAINTEXT",
    "auto.offset.reset": "beginning"
}

# New field is a vairable containing the following information  new_field = { "sasl.username":"my_user","sasl.password": "my_password"}

new_field = {{  var.json.new_field}}
config_dict = extra | new_field