search

Mintplex-Labs / anything-llm

The all-in-one Desktop & Docker AI application with built-in RAG, AI agents, and more.
https://anythingllm.com
MIT License
27.18k stars 2.72k forks source link

Support Mlflow as LLM-Provider #486

Open essamabas opened 11 months ago

essamabas commented 11 months ago

is it planned to support MlFlow as LLM-Provider ?

timothycarambat commented 11 months ago

At this time, no.

MLFlow is for training models as I understood. Do they provide a normalized API for running inferencing as well? You are the first request for MLFlow we have gotten, more support would prioritize this addition 👍

essamabas commented 11 months ago

Yes, MLFlow is used or training/ evaluationg as well as serving Models. It has unified APIs to return Models with their End-Points https://mlflow.org/docs/latest/llms/deployments/api.html#/

I can work on an extension and provide with pull request.

Jayasankar-kk commented 6 months ago

are there plans to add mlflow as an llm provider now