I have an airflow pipeline which creates a dataproc cluster using this airflow operator, submits a spark job, and then deletes the cluster with a similar operator as creating. Could I get this type of behavior added as kestra tasks?
In the slack space, a great suggestion was given to just use the GCP CLI which can help me try to recreate this pipeline, but for wider adoption, a task like the airflow operator would make it easier.
Feature description
I have an airflow pipeline which creates a dataproc cluster using this airflow operator, submits a spark job, and then deletes the cluster with a similar operator as creating. Could I get this type of behavior added as kestra tasks?
In the slack space, a great suggestion was given to just use the GCP CLI which can help me try to recreate this pipeline, but for wider adoption, a task like the airflow operator would make it easier.