gbif / pipelines

Pipelines for data processing (GBIF and LivingAtlases)
Apache License 2.0
40 stars 28 forks source link

K8s: Use Airflow runner for pipelines #1058

Closed muttcg closed 1 month ago

muttcg commented 2 months ago

Pipelines uses custom K8s runner which is redundant and requires extra support of one more library, downloads service uses Airflow as a runner and Pipelines can reuse the same Airflow client