getindata / kedro-airflow-k8s

Kedro Plugin to support running pipelines on Kubernetes using Airflow.
https://kedro-airflow-k8s.readthedocs.io
Apache License 2.0
29 stars 11 forks source link

Ensure pipeline works when git not initialized #110

Open em-pe opened 3 years ago

em-pe commented 3 years ago

Currently we use kedro session store to properly tag airflow DAG

https://github.com/getindata/kedro-airflow-k8s/blob/3b30df2a831cd5672f6630586d0bae4430a41973/kedro_airflow_k8s/template.py#L96

This causes build failure when git is not initialized in a project.

My suggestion is to obtain git sha the following way: IF Check if KEDRO_CONFIG_COMMIT_ID env variable is set and then use it, ELSE IF Check the session store for git info and if available use it ELSE set git sha to "UNKNOWN"