uc-cdis / cloud-automation

Automation for standing up Gen3 commons
Apache License 2.0
41 stars 68 forks source link

Roll Spark Service in Gen3 Roll All #2543

Open EliseCastle23 opened 2 months ago

EliseCastle23 commented 2 months ago

editing spark deployment in "kube-roll-all" to roll Spark if the ETL job is not running

Improvements

Spark was not getting rolled for releases when using "gen3 roll all" since the logic would only roll spark if the deployment didn't exist already. We want to change the functionality to roll the Spark service as long as there are no actively running ETL jobs.