databricks / mlops-stacks

This repo provides a customizable stack for starting new ML projects on Databricks that follow production best-practices out of the box.
https://docs.databricks.com/en/dev-tools/bundles/mlops-stacks.html
Apache License 2.0
425 stars 143 forks source link

Replace MLflow Recipes as default option with Delta Tables #93

Closed arpitjasa-db closed 1 year ago

arpitjasa-db commented 1 year ago

We update MLOps Stacks to use Delta Tables by default instead of MLflow Recipes.

Tested using generated stack here: https://github.com/databricks/test-delta-stacks, and confirmed conflicting options cannot be selected:

Screenshot 2023-07-28 at 11 06 51 PM
mingyu89 commented 1 year ago

It looks good to me. That's a lot of condition changes.

As one more step of sanity check, could you generate the mlflow recipes project and fs project using both master and your branch? Then do a diff of the two projects? Also do a diff of the training notebooks since they are renamed now.

arpitjasa-db commented 1 year ago

It looks good to me. That's a lot of condition changes.

As one more step of sanity check, could you generate the mlflow recipes project and fs project using both master and your branch? Then do a diff of the two projects? Also do a diff of the training notebooks since they are renamed now.

Here's Feature Store: https://github.com/databricks/test-delta-stacks/pull/4 and MLflow Recipes: https://github.com/databricks/test-delta-stacks/pull/5 when compared to Delta Tables.

Here's Feature Store: https://github.com/databricks/test-delta-stacks/pull/6 and MLflow Recipes: https://github.com/databricks/test-delta-stacks/pull/7 when compared to the current version pulled from the main branch of Stacks at this time.

mabreuortega commented 1 year ago

I'm having issues with this upgrade, cookiecutter no longer works and the suggested databricks cli version is not available

mingyu89 commented 1 year ago

@mabreuortega The databricks cli version is available here https://github.com/databricks/cli/releases/tag/v0.204.0

mingyu89 commented 1 year ago

@mabreuortega Created PR to explicitly add the link to databricks cli installation doc and releases page https://github.com/databricks/mlops-stack/pull/100