search

stikkireddy / mlflow-extensions

Deploy models quickly to databricks via mlflow based serving infra.
https://stikkireddy.github.io/mlflow-extensions/
Apache License 2.0
19 stars 11 forks source link

[FEATURE] Integrate with Mosaic AI gateway #58

Closed sathishgang-db closed 2 months ago

sathishgang-db commented 2 months ago

Is your feature request related to a problem? Please describe. If there's a way to take the model endpoint and make it be managed entirely using the mosaic AI gateway, that would open up all of the features from that product - usage tracking, payload logging, guardrails, ratelimiting etc. Would be an amazing add on to any SoTA model supported

This would allow users to cleanly compare between new supported models (maybe domain specific) and other frontier proprietary provider models by setting up routing rules as well. Will simplify model selection and allow for science based reasoning for the choices made during evals.

stikkireddy commented 2 months ago

completed as part of v0.15.0 and documented here: https://stikkireddy.github.io/mlflow-extensions/getting-started/ezdeploylite/#registering-into-mosaic-ai-gateway