As discussed with @DSuveges, under the current implementation, if a locally running Airflow instance is interrupted (crashes, laptop goes to sleep etc.), there is a possibility that the Dataproc cluster will continue running indefinitely, consuming funds, because the Airflow instance isn't going to be there to delete it.
While this hasn't happened yet, and in the long run orchestration will be migrated to Google Cloud Composer, it seems sensible to add a failsafe to delete an idle cluster after a certain amount of minutes.
As discussed with @DSuveges, under the current implementation, if a locally running Airflow instance is interrupted (crashes, laptop goes to sleep etc.), there is a possibility that the Dataproc cluster will continue running indefinitely, consuming funds, because the Airflow instance isn't going to be there to delete it.
While this hasn't happened yet, and in the long run orchestration will be migrated to Google Cloud Composer, it seems sensible to add a failsafe to delete an idle cluster after a certain amount of minutes.