databricks / mlops-stacks

This repo provides a customizable stack for starting new ML projects on Databricks that follow production best-practices out of the box.
https://docs.databricks.com/en/dev-tools/bundles/mlops-stacks.html
Apache License 2.0
456 stars 156 forks source link

Add env prefix to batch inference output table name #87

Closed mingyu89 closed 1 year ago

mingyu89 commented 1 year ago

If the staging env and dev env are referring to the same workspace, the staging batch inference workflow and the dev batch inference workflow could try to write to the same table at the same time and throw concurrent modification error.

To avoid concurrent modification between different workflows, we want to add the env prefix to the output table.