search

apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
https://airflow.apache.org/
Apache License 2.0
37.15k stars 14.32k forks source link

Make Airlfow integration with Open-Telemetry "pluggable" #20778

Closed potiuk closed 2 years ago

potiuk commented 2 years ago

We should be able to "plug-in" the open-telemetry integration in Airflow. This should be possible to be done on several layers:

1) we should be able to integrate different exporters by just enabling them (Jaeger, Console, ....). Currently we know that we have to some Gunicorn/Fork workarounds (and write some small code snippets specific per exporter) but ideally we should be able to just configure the exporter

2) Some basic exporters should be runnable via breeze env ("--integration" flag) - this way we should be able to test and enable open-telemetry integration as an optional component during airflow development

Melodie97 commented 2 years ago

Ok, I’ll work on this

eladkal commented 2 years ago

Closing in favor of AIP-49 OpenTelemetry Support for Apache Airflow