[FR] Allow not having a "latest" alias by default

DavidSlayback commented 1 month ago

Willingness to contribute

Yes. I can contribute this feature independently.

Proposal Summary

I propose to either:

Allow reconfiguring the UI of an MLFlow instance to treat some other alias the same way it treats the "latest" alias (e.g., make a different alias the first one seen on the screen below instead of putting it in the dropdown)
Allow disabling the automatic aliasing with "latest" for an MLFlow instance such that the logging user can manually control which model versions are actually promoted as "latest"

Obviously I don't want either of these as defaults. But to show an example of what I'm trying to do, the below image is the registry for one model. The one we actually care about is version 5 ("last_trained") and is hidden behind a dropdown, even though the last version we created was from a hyperparameter tuning run ("last_tuned" and "latest"). The reason both of these are under the same registry is so that it's trivial for any training run to grab the most recent optimized set of hyperparameters

Motivation

What is the use case for this feature?

I understand that in most use cases, having the last version of a registered model be assigned the "latest" alias and moved to the front of the line for sourcing versions is useful behavior. In my case, however, it's often undesired to do this because the latest version might actually be from a hyperparameter tuning run that trained on different data or from a training run that isn't trusted yet.

Why is this use case valuable to support for MLflow users in general?

Having more options to reconfigure the registry interface can make it easier to onboard users and cater to more organizations that would like to use MLFlow

Why is this use case valuable to support for your project(s) or organization?

We often create versions of the same model that we don't necessarily intend to use directly, and we also often create model versions that we want to validate via separate processes before they are allowed to be used for inference. We can do this programmatically, but the friction in the UX of a heavily-populated MLFlow server can be difficult to deal with

Why is it currently difficult to achieve this use case?

Obviously it's easy to get the right alias programmatically when running actual code, and I just have different "latest_X" aliases to accomplish it. But the UI/UX experience still heavily favors the "latest" alias, making it more difficult for my team members to find the actual version they're looking for when training.

Alternatively I could also hide all the untrusted/tuning runs behind different registry names, but then it creates another UX issue where the size of the registry that the user has to search through doubles/triples

Details

No response

What component(s) does this bug affect?

[ ] area/artifacts: Artifact stores and artifact logging
[ ] area/build: Build and test infrastructure for MLflow
[ ] area/deployments: MLflow Deployments client APIs, server, and third-party Deployments integrations
[ ] area/docs: MLflow documentation pages
[ ] area/examples: Example code
[X] area/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registry
[ ] area/models: MLmodel format, model serialization/deserialization, flavors
[ ] area/recipes: Recipes, Recipe APIs, Recipe configs, Recipe Templates
[ ] area/projects: MLproject format, project running backends
[ ] area/scoring: MLflow Model server, model deployment tools, Spark UDFs
[ ] area/server-infra: MLflow Tracking server backend
[ ] area/tracking: Tracking Service, tracking client APIs, autologging

What interface(s) does this bug affect?

[X] area/uiux: Front-end, user experience, plotting, JavaScript, JavaScript dev server
[ ] area/docker: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
[ ] area/sqlalchemy: Use of SQLAlchemy in the Tracking Service or Model Registry
[ ] area/windows: Windows support

What language(s) does this bug affect?

[ ] language/r: R APIs and clients
[ ] language/java: Java APIs and clients
[ ] language/new: Proposals for new client languages

What integration(s) does this bug affect?

[ ] integrations/azure: Azure and Azure ML integrations
[ ] integrations/sagemaker: SageMaker integrations
[ ] integrations/databricks: Databricks integrations

WeichenXu123 commented 1 month ago

Allow reconfiguring the UI

How to design the configuration UI ? e.g., clicking a button, prompting a form to select the items to be shown in the page ?

Allow disabling the automatic aliasing with "latest"

How to do this ? Adding a environmental variable configuration ?

CC @daniellok-db What do you think of the idea ? I think it makes sense but need to do a bit more UI design.

github-actions[bot] commented 4 weeks ago

@mlflow/mlflow-team Please assign a maintainer and start triaging this issue.

mlflow / mlflow