search

mlc-ai / mlc-llm

Universal LLM Deployment Engine with ML Compilation
https://llm.mlc.ai/
Apache License 2.0
18.69k stars 1.52k forks source link

[Feature Request] [REST] Add a /v1/models REST API endpoint #1400

Closed CamilleHbp closed 6 months ago

CamilleHbp commented 9 months ago

🚀 Feature

It would be nice to get a /v1/models REST endpoint with a behaviour similar to OpenAI or oobabooga/Text-Generation-WebUI: lists models, currently loaded model first.

Motivation

I am writing a connector for MLC-LLM in SillyTavern. There is currently no way to get a status report and a model list from MLC-LLM. This would be wonderful to increase the discovery and use of the project since SillyTavern is used by many people. It would also allow us to use existing models without having to remember the exact name of the model and if we installed it in MLC-LLM.

Alternatives

I don't know of any?

Additional context

None.

Kartik14 commented 6 months ago

This is supported now. Closing this issue.