Deployed model from outside the cluster or Pipeline step

[Barteus]

Reproduce:

1. Install Kubeflow 1.6
2. Deploy the model in the user namespace
3. Create a task to access the API. -> Result 403.

I can call the same model from a notebook or from a manually deployed Ubuntu Pod.

Notebook with steps: https://github.com/canonical/ai-ml-demos/blob/main/talking-jellyfish/models/chatbot/deploy-pipeline.ipynb

Possible issue: Istio sidecar is not added to the step

[Barteus]

Exposing the model from Kubernetes where Charmed Kubeflow is deployed cluster using Virtual Service also does not work.

If I used the cluster with only seldon-core and istio (gateway + pilot) deployed it works fine.

Example Deploy model:

kubectl apply -f - << END
apiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
metadata:
  name: chatbot
spec:
  name: chatbot
  predictors:
  - componentSpecs:
    - spec:
        containers:
        - name: classifier
          image: bponieckiklotz/jellyfish.chatbot:dev@sha256:a1ce5fcdc31e3c393eb47e18245bebc789aa6879f54611471c0a57f0a440b2e4
          securityContext:
            allowPrivilegeEscalation: false
            runAsUser: 0
    graph:
      name: classifier
    name: default
    replicas: 1
END

Add Virtual Service:

cat <<EOF | kubectl create -f -
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: chatbot-server
  namespace: admin
spec:
  gateways:
    - kubeflow/kubeflow-gateway
  hosts:
    - '*'
  http:
    - match:
        - uri:
            prefix: /model/chatbot/
      rewrite:
        uri: /
      route:
        - destination:
            host: chatbot-default.default.svc.cluster.local
            port:
              number: 8000
EOF

Adjust config for not authenticated URLs:

juju config oidc-gatekeeper skip-auth-urls="/model/chatbot/"

Call the endpoint:

curl  -s http://10.64.140.43.nip.io/model/chatbot/api/v0.1/predictions   \
  -H "Content-Type: application/json"   \
  -d '{"data":{"ndarray":["1", "HiDNS"]}}'

[ca-scribner]

I don't think this is a bug, I think you're missing the authentication step. The curl doesn't have authentication, so i think you're redirected into the auth loop.

This unmerged PR has an example notebook showing what I mean. Does that solve you're problem?

[Barteus]

If you use notebooks then it works because you also have an istio sidecar. This will work.

If authentication is wrong you will get redirected to dex/login page. I have added the link to non-auth links to simplify this task. You can also try to add cookies from your browser to the request.