apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
https://airflow.apache.org/
Apache License 2.0
35.21k stars 13.76k forks source link

Kafka Operators - Add context to ConsumeFromTopicOperator's apply_function #40416

Open sheinbergon opened 4 days ago

sheinbergon commented 4 days ago

boring-cyborg[bot] commented 4 days ago

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst) Here are some useful points:

sheinbergon commented 3 days ago

What will happen when someone updates the provider version, and has an apply_function callable that processes a single argument (without **kwargs)?

IMHO it will fail, so we need to create a new major release or make it backward compatible.

I guess I can use inspect to test if the callbable expects that context and decide to call it or not based on the inspection result. Would that be preferable in your opinion?