The default Jinja2-Template works well as a base to get started on using kedro with Airflow. However, the fact that a few things are hardcoded in the plugin/template makes it hard to modify a variety of things. I would propose a couple of adjustments in order to allow for more customisability:
Provide CLI Arguments for folder and name of jinja2 Templates
Prefix/Postfix the name of the resulting dag files with the pipeline name if a pipeline was given
I have already implemented those in a branch and was curious if this would be a welcomed PR.
Context
In a recent project I found myself constantly having to adjust the resulting dag files as I was experimenting with a variety of things. I ended up replacing the template file in the site-packages folder at some point which struck me as too hacky. The above changes would allow users to provide their own Jinja2 Templates as well as generating Airflow Dags for multiple pipelines without having to manually rename the output files.
Description
The default Jinja2-Template works well as a base to get started on using kedro with Airflow. However, the fact that a few things are hardcoded in the plugin/template makes it hard to modify a variety of things. I would propose a couple of adjustments in order to allow for more customisability:
I have already implemented those in a branch and was curious if this would be a welcomed PR.
Context
In a recent project I found myself constantly having to adjust the resulting dag files as I was experimenting with a variety of things. I ended up replacing the template file in the
site-packages
folder at some point which struck me as too hacky. The above changes would allow users to provide their own Jinja2 Templates as well as generating Airflow Dags for multiple pipelines without having to manually rename the output files.Possible Implementation
Add two more click options (names just examples):
Modify the loader and template respectively:
Adjust the dag_filename if necessary:
Looking forward to some feedback on this suggestions.