We use a gateway between the CI build and the CD deployment into prod environments. It would be cleaner for us to build a JAR using SBT during the build process to ensure the build is successful and the tests pass, and then deploy the resulting JAR artefact into multiple Databricks clusters later. Also separating the deployment of data into environments could be helpful too.
We use a gateway between the CI build and the CD deployment into prod environments. It would be cleaner for us to build a JAR using SBT during the build process to ensure the build is successful and the tests pass, and then deploy the resulting JAR artefact into multiple Databricks clusters later. Also separating the deployment of data into environments could be helpful too.
Happy to contribute to the refactoring.