databricks-demos / dbdemos

Demos to implement your Databricks Lakehouse
Other
255 stars 80 forks source link

End-to-End MLOps Demo (Is this the recommended approach?) #83

Open VirajVaitha123 opened 7 months ago

VirajVaitha123 commented 7 months ago

Hi,

Firstly, thank you for a streamlined approach to testing and running a demo, it was great!.

I just want to clarify that isn't the recommended MLOps workflow shown in your documentation. The Big Book of MLOps, or MLOps Workflow Blog show a different approach. This is like a model promotion MLOps workflow achieved in a single environment?, as opposed to code deployment, where we are promoting code from dev, staging, prod, and CI/CD pipelines orchestrate the relevant workflows?

https://docs.databricks.com/en/machine-learning/mlops/mlops-workflow.html#development-stage

Is there an example of the approach you recommended?

QuentinAmbard commented 7 months ago

Hi, That's a very valid feedback, thanks for sharing. The mlops demo is a bit outdated and we have a PR open to move it to UC, which will improve the cycle with dev/prod and multiple catalog. That being said, we want the demo to remain simple so we'll use a simple catalog, but add comments to explain the overall idea. We'll release the new version soon, and I'll update the ticket asap :)

aelhelouDB commented 7 months ago

Thank you for the feedback, we should probably rename this demo to ModelOps or something which indeed reflects what the demo is doing which is just a "deploy/promote model" only as part of single env (can be dev or prod). That being said we will update this demo with all new UC and model serving capabilities to showcase all of the platform capabilities.

We are working on another demo showcasing the "deploy code" approach you're referring to using our MLOps stack and we encourage you to try things out yourself and give us feedback/create issues.