astronomer / airflow-provider-kafka

A provider package for kafka
Apache License 2.0
37 stars 16 forks source link

Add `config` attributes to `template_fields` #23

Closed oboki closed 1 year ago

oboki commented 1 year ago

I wish to manage kafka_config attribute as template_fields.

I want to have different bootstrap.servers value depending on the deployment environment, such as develop, test, and production. It would be useful while managing different values in the Variable store for each deployment environment with the same DAG code.

For example, in the ProduceToTopicOperator, I can use:

kafka_config={"bootstrap.servers": "{{ var.value.get('kafka__bootstrap_servers')}}"},
dylanbstorey commented 1 year ago

Thanks for you contribution - it will go out in the next release.