kubeflow / manifests

A repository for Kustomize manifests
Apache License 2.0
832 stars 889 forks source link

Kubeflow & ModelMesh-Serving Compatibility #2925

Open muratyarali opened 5 days ago

muratyarali commented 5 days ago

Validation Checklist

Version

1.9

Describe your issue

I deployed ModelMesh-Serving in a Kubeflow K8S cluster (kubeflow version: v1.9.1-RC.2) and created an InferenceService (ISVC) with modelmesh as follows:

kubectl apply -f - <<EOF
apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: example-sklearn-isvc
  namespace: modelmesh-serving
  annotations:
    serving.kserve.io/deploymentMode: ModelMesh
spec:
  predictor:
    model:
      modelFormat:
        name: sklearn
      storageUri: "gs://kfserving-examples/models/sklearn/1.0/model"
EOF

Annotation is used to select the model mesh for deploying an ISVC. When I create an ISVC with KServe (without annotation), I can view it on the KServe Endpoints tab in the Kubeflow UI, and clicking on it reveals the details.

However, when I create the ISVC with ModelMesh-Serving, I can see it listed in the KServe Endpoints tab, but clicking on it does not display the details and instead shows an error:

[404] The requested resource could not be found in the API Server https://www.example.com/kserve-endpoints/api/namespaces/modelmesh-serving/revisions/undefined

The Kubeflow-UI is looking for resources at /kserve-endpoints/api/namespaces/modelmesh-serving/revisions/ which don't exist for ModelMesh.

Can this be fixed or extended on the Kubeflow or ModelMesh-Serving side?

Steps to reproduce the issue

  1. I checked this with Kubeflow
  2. I checked this ModelMesh-Serving

Put here any screenshots or videos (optional)

No response

juliusvonkohout commented 5 days ago

Probably yes, but what we need is someone willing to create PRs to fix it.