Closed OvervCW closed 6 months ago
@OvervCW there shouldn't be anything stopping you using these other kinds of models with Triton, would be great if you could try it out!
Hopefully all that should be needed is to:
triton-2.x
ServingRuntime
spec to include additional supportedModelFormats
as neededautoSelect
to false
in the right places in those other runtimes)@rafvasq -- since you updated the Triton serving runtime last year, it might be worth trying this out and documenting it. WDYT?
@ckadner I confirmed that with a few changes, Triton's able to deploy those models too. Think it'd be worth updating the docs to reflect that which includes a couple of small updates to the example models via https://github.com/kserve/modelmesh-minio-examples/pull/7.
ModelMesh currently uses the MLServer runtime to serve sklearn, xgboost, and lightgbm models. However, it seems like recent versions of Triton now support all of those model types as well.
Does that mean that ModelMesh could support all types of models using a single runtime? And if so, would it be easy to change the adapter to handle this?