[FR] trigger deployment of models via MLFLOW UI

[echarso]

Thank you for submitting a feature request. Before proceeding, please review MLflow's Issue Policy for feature requests and the MLflow Contributing Guide.

Please fill in this feature request template to ensure a timely and thorough response.

Willingness to contribute

The MLflow Community encourages new feature contributions. Would you or another member of your organization be willing to contribute an implementation of this feature (either as an MLflow Plugin or an enhancement to the MLflow code base)?

• [ ] Yes. I can contribute this feature independently.
• [X] Yes. I would be willing to contribute this feature with guidance from the MLflow community.
• [ ] No. I cannot contribute this feature at this time.

Proposal Summary

In order to have Mlflow as a central component in the model deployment and governance process, we would like to trigger model deployments by also using the MLFLOW UI and using the mlflow model stage button .

Right now this is not available and the work around is to have a small daemon running all the time, pulling information from the model registry and deploy or undeploy models accordingly . If we could have an even triggered from MLFLOW server when a model state transition is happening then we could save a lot of time .

I could be willing to investigate how much effort this is and potentially implement this. What I would like is some direction on where in the code to look for propagating those events from the GUI to the backend server.

Motivation

• What is the use case for this feature?
• Why is this use case valuable to support for MLflow users in general?
• Why is this use case valuable to support for your project(s) or organization?
• Why is it currently difficult to achieve this use case? (please be as specific as possible about why related MLflow features and components are insufficient)

What component(s), interfaces, languages, and integrations does this feature affect?

Components

• [ ] area/artifacts: Artifact stores and artifact logging
• [ ] area/build: Build and test infrastructure for MLflow
• [ x] area/docs: MLflow documentation pages
• [ ] area/examples: Example code
• [ ] area/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registry
• [ ] area/models: MLmodel format, model serialization/deserialization, flavors
• [ ] area/projects: MLproject format, project running backends
• [ ] area/scoring: Local serving, model deployment tools, spark UDFs
• [x ] area/server-infra: MLflow server, JavaScript dev server
• [ ] area/tracking: Tracking Service, tracking client APIs, autologging

Interfaces

• [x ] area/uiux: Front-end, user experience, JavaScript, plotting
• [ ] area/docker: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
• [ ] area/sqlalchemy: Use of SQLAlchemy in the Tracking Service or Model Registry
• [ ] area/windows: Windows support

Languages

• [ ] language/r: R APIs and clients
• [ ] language/java: Java APIs and clients
• [ ] language/new: Proposals for new client languages

Integrations

• [ ] integrations/azure: Azure and Azure ML integrations
• [ ] integrations/sagemaker: SageMaker integrations
• [ ] integrations/databricks: Databricks integrations

Details

(Use this section to include any additional information about the feature. If you have a proposal for how to implement this feature, please include it here. For implementation guidelines, please refer to the Contributing Guide.)

[dmatrix]

@echarso Thanks for filing this feature enhancement. Can you please take some moments to answer questions under Motivation? I know bits of it are articulated in summary, but nice to have them as part of the answers to those questions.

I could be willing to investigate how much effort this is and potentially implement this.

Excellent! This seems like a greater effort as there are many changes and will require touching many aspects of the code. It will require some design and review from the respected committers. Because MLflow models can be deployed to myriad environments, you'll have to account for those use cases. Or minimally, start with a common deployment environment as a the first scenario; then add others later.

What I would like is some direction on where in the code to look for propagating those events from the GUI to the backend server.

Right now, this is not available, and the workaround is to have a small daemon running all the time, pulling information from the model registry, and deploy or undeploy models accordingly. If we could have an even triggered from MLFLOW server when a model state transition is happening, we could save a lot of time.

It seems like you are suggesting to implement two features here:

1. Capture model state transitions–staging or production or archive – as events instead of running daemon that pulls the model registry.
2. Once transitioned, enable a "Deploy" button in the UI

The first will require a serverside deployment plugin that enables a certain action to be executed when a model transitions. This could be implemented as a webhook. And the second is a result of the first, which may be interesting, as you will want to change the UI or enable the Deploy button dynamically.

I'll have some team members comment on this feature enhancement request cc @sueann @smurching @xanderwebs

[echarso]

super! Thanks for answer.

To your first comment .

'Excellent! This seems like a greater effort as there are many changes and will require touching many aspects of the code. It will require some design and review from the respected committers. Because MLflow models can be deployed to myriad environments, you'll have to account for those use cases. Or minimally, start with a common deployment environment as a the first scenario; then add others later. ' I was thinking to capture the event from the GUI passed it to the back end , and then send a rest call with the event to a url . The url could be passed as a configuration parameter ( deployment service url for example) . Behind the url will be a service acting on the event . That service can be implemented on a use case specific basis and does not need to be part of the MLFLOW source code. Of course template implementations of the service could be added but I was not seen it as a core thing.

To your second part to include a deploy button . I was not thinking to have a deploy button , but rather a deployment to be triggered when a model is transitioned to production stage for example

[dmatrix]

@echarso IMHO, I believe for the first case, you are evaluating or examining what we think would be a general mechanism to provide a server-side plugging interface, similar to what we have for deployment plugins for model registry state transitions.

The idea there is the developer will implement abstract methods that will invoke user-defined webhooks. The server would check if there are any registered callbacks upon an event, such as model state transition, and invoke them. This callback could be a web request to a service.

We don't have that server-side mechanism yet, and we are happy to get community contributions. It is still better to design this as a general plugin architecture as we have for deployment plugins. As such, others can write/contribute their own server-side webhooks for CI/CD services or custom services needed or acted upon model state transitions.

This would be a huge benefit to the community.

[alexott]

@dmatrix I have this idea in my head for a some time already, I just need to find time to prepare proposal - I maybe need several days for that, hopefully over this weekend.

[echarso]

@dmatrix your second paragraph describes exactly what i would like to do . In case you need help implementing this I would like to be part of the work.

[hugowschneider]

Hi @echarso and @dmatrix,

I was looking into this FR and I am very interested in following up on that and if possible contribute :), have any of you started any implementation?

I like the proposed approach to implemented is similar to the plugins. I was hoping to achieve with this a 2-step approach.

1. A callable would give feedback, based on the information stored in MLFlow, if a model can be moved from one stage to another. This would be an automated feedback besides the manual one.
2. After approving this "move" a second callable would trigger the model deploy.

Does this make sense?

[echarso]

I have not started any development on that. My time is a little bit limited right now. What we have done for now is to have a pull service to listen to model registry updates.

[shaikmoeed]

@echarso & @dmatrix I am also interested in this feature. Is there any possibility to add this feature? I am willing to contribute with community guidance :)

[carlpihl1]

Any updates? @dmatrix's idea of introducing server-side plugins to be able to act upon e.g. model state transitions sounds fantastic. Being able to trigger deployments based on model state transitions is the missing piece in my MLOps puzzle :)

[dataubc]

I have a keen interest in this feature as well. Can it be added? I'm open to helping out

[RajezMariner]

Could we simply import a simple callback function when the model transition happens. Rest, the client/backend deployment teams could take the initiative for the same.

One use case:

1. We may have an integration with airflow/github repo - call their DAG or github build through the custom callbacks. I guess, this approach would work. Cause, I am thinking of generating certain events like this.

[spicoflorin]

Hello! Any updates on this request?

[ishan-modi]

Any Updates on this request ??

[vyadav-wiq]

I think the suggestion made by @hugowschneider in https://github.com/mlflow/mlflow/issues/4219#issuecomment-1080319602 would be a simple generalised solution, similar to webhooks in GitHub