search

etsy / boundary-layer

Builds Airflow DAGs from configuration files. Powers all DAGs on the Etsy Data Platform
Apache License 2.0
262 stars 58 forks source link

Add do_xcom_push to BaseOperator config (default plugin) #120

Open peleyal opened 3 years ago

peleyal commented 3 years ago

do_xcom_push is the right way to use xcom in Airflow (xcom_push on k8s pod operator doesn't actually work on newer versions).

peleyal commented 3 years ago

@vchiapaikeo, I added this one in our plugin, but I think it can help other developers as well.

Thanks!

vchiapaikeo commented 3 years ago

Hi @peleyal - we're going to start slowly working our way to Airflow 2.0 - hopefully that will happen by EOY. Until then though, I'd prefer not to merge this to master. We could consider merging this to a separate branch though.

peleyal commented 3 years ago

Hi @peleyal - we're going to start slowly working our way to Airflow 2.0 - hopefully that will happen by EOY. Until then though, I'd prefer not to merge this to master. We could consider merging this to a separate branch though.

Thanks for the quick response.

I'm pretty sure it's not a feature of 2.0, as we are using Google Cloud Composer, and it runs with an older version Airflow.

Totally up to you, feel free to close this one. I already merged it to our private plugin... I just wanted to avoid a case that other developers will be stuck on it (like I was).

Thanks!