search

kserve / modelmesh-serving

Controller for ModelMesh
Apache License 2.0
204 stars 114 forks source link

Add Support for Swagger UI #356

Open taneem-ibrahim opened 1 year ago

taneem-ibrahim commented 1 year ago

Is your feature request related to a problem? If so, please describe. Currently the kserve inference swagger UI does not work with model mesh. The swagger UI is very useful to developers because it allows interacting with the APIs in a visual way on the browser.

In case of model mesh we need a way to provide a swagger UI for each serving runtime so that each inference request are routed to right set of replicas for a given model.

ckadner commented 1 year ago

Should this be a REST or gRPC based API? I would think REST + JSON would be simpler and easier to use and we could probably add the necessary code in the rest-proxy repository.

@taneem-ibrahim -- Why would we need a swagger UI for each serving runtime instead of relying on modelmesh to route to any replica of the model?

taneem-ibrahim commented 1 year ago

Yes Rest+JSON should be fine. Relying on modelmesh to route is okay too. Thank you!