inferencing pytorch with curl is returning a HTTP/1.1 404 Not Found error.

[taylanates24]

/kind bug

What steps did you take and what happened: I installed kfserving with a minikube kubernetes cluster following the guide https://github.com/kubeflow/kfserving/blob/master/README.md. I used kfserving master branch and Kubernetes v1.18.3. And then, I followed the steps on https://github.com/kubeflow/kfserving/tree/master/docs/samples/pytorch I successfully created inferenceservice.serving.kubeflow.org/pytorch-cifar10. But inference with curl was not successful. I used the following names:

MODEL_NAME=pytorch-cifar10
INPUT_PATH=@./input.json
SERVICE_HOSTNAME=$(kubectl get inferenceservice pytorch-cifar10 -o jsonpath='{.status.url}' | cut -d "/" -f 3)
export INGRESS_HOST=$(minikube ip)
export INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].nodePort}')

Because the EXTERNAL-IP value is none (or perpetually pending), my environment does not provide an external load balancer for the ingress gateway I use the following command:

curl -v -H "Host: ${SERVICE_HOSTNAME}" -d $INPUT_PATH http://${INGRESS_HOST}:${INGRESS_PORT}/v1/models/$MODEL_NAME:predict

and it gives the following output:

   Trying 192.168.39.161...
* TCP_NODELAY set
* Connected to 192.168.39.161 (192.168.39.161) port 31603 (#0)
> POST /v1/models/pytorch-cifar10:predict HTTP/1.1
> Host: 
> User-Agent: curl/7.58.0
> Accept: */*
> Content-Length: 110681
> Content-Type: application/x-www-form-urlencoded
> Expect: 100-continue
> 
< HTTP/1.1 100 Continue
< HTTP/1.1 404 Not Found
< date: Fri, 14 Aug 2020 14:09:58 GMT
< server: istio-envoy
< connection: close
< content-length: 0
< 
* we are done reading and this is set to close, stop send
* Closing connection 0

I didn't understand what the problem is, and how I can solve this problem. Please help me, thank you in advance.

Anything else you would like to add:

I created a namespace as:

kubectl create namespace pytorch-test

after that;

kubectl apply -f pytorch.yaml -n pytorch-test

when I run the following command:

kubectl get inferenceservices pytorch-cifar10 -n pytorch-test

It gives the output:

NAME              URL                                               READY   DEFAULT TRAFFIC   CANARY TRAFFIC   AGE
pytorch-cifar10   http://pytorch-cifar10.pytorch-test.example.com   True    100                                15m

So, I am not sure if the url "http://${INGRESS_HOST}:${INGRESS_PORT}/v1/models/$MODEL_NAME:predict" is true. What should the url be? I think the problem is the wrong url.

What did you expect to happen: I expect to see the predictions after the curl command.

Environment:

• Istio Version: 1.6.8
• Knative Version:0.15.0
• KFServing Version:v0.4.0
• Kubeflow version: I didn't find
• Kfdef:[k8s_istio/istio_dex/gcp_basic_auth/gcp_iap/aws/aws_cognito/ibm]
• Minikube version:v1.12.3
• Kubernetes version: (use kubectl version):v1.18.7
• OS (e.g. from /etc/os-release): NAME="Ubuntu" VERSION="18.04.4 LTS (Bionic Beaver)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 18.04.4 LTS" VERSION_ID="18.04" HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" VERSION_CODENAME=bionic UBUNTU_CODENAME=bionic

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the labels:

Label	Probability
area/inference	0.74

Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback! Links: app homepage, dashboard and code for this bot.

[issue-label-bot[bot]]

Issue Label Bot is not confident enough to auto-label this issue. See dashboard for more details.

[taylanates24]

I solved the issue by updating the SERVICE_HOSTNAME from SERVICE_HOSTNAME=$(kubectl get inferenceservice pytorch-cifar10 -test -o jsonpath='{.status.url}' | cut -d "/" -f 3) to SERVICE_HOSTNAME=$(kubectl get inferenceservice pytorch-cifar10 -n pytorch-test -o jsonpath='{.status.url}' | cut -d "/" -f 3)

[yuzisun]

@taylanates24 Do you think we should make the instruction more clear?

[Sangauppe]

INGRESS_HOST}:${INGRESS_PORT}/v1/models/$MODEL_NAME:predict

• Trying 127.0.0.1:8080...
• Connected to localhost (127.0.0.1) port 8080 (#0)

  POST /v1/models/cifar10:predict HTTP/1.1 Host: torch-transformer.default.example.com User-Agent: curl/7.81.0 Accept: / Content-Length: 3394 Content-Type: application/x-www-form-urlencoded

• Mark bundle as not supporting multiuse < HTTP/1.1 500 Internal Server Error < content-length: 69 < content-type: application/json < date: Thu, 25 May 2023 09:49:30 GMT < server: istio-envoy < x-envoy-upstream-service-time: 4 <
• Connection #0 to host localhost left intact {"error":"AttributeError : 'bytes' object has no attribute 'inputs'"} @all I am getting this error while working with grpc torch-transformer, and same with REST torch-transformer model can anyoen help me with it?

[nguyenthai0107]

Hello @Sangauppe @yuzisun
I facing same issue with status code 500 error when call to sklearn-iris `curl: (6) Could not resolve host: POST

• Trying 10.xxx.xx.xxx:80...
• Connected to sklearn-iris.kserve-test.svc.cluster.local (10.104.26.188) port 80 (#1)

  POST /v1/models/sklearn-iris:predict HTTP/1.1 Host: sklearn-iris.kserve-test.svc.cluster.local User-Agent: curl/7.80.0 Accept: / Content-Length: 30 Content-Type: application/x-www-form-urlencoded

• Mark bundle as not supporting multiuse < HTTP/1.1 500 Internal Server Error < content-length: 211 < content-type: application/json < date: Sat, 20 Jan 2024 08:11:15 GMT < server: istio-envoy < x-envoy-upstream-service-time: 5 <
• Connection #1 to host sklearn-iris.kserve-test.svc.cluster.local left intact {"error":"Expected 2D array, got scalar array instead:\narray=nan.\nReshape your data either using array.reshape(-1, 1) if your data has a single feature or array.reshape(1, -1) if it contains a single sample."}` @Sangauppe , how did you resolved it ? Thanks

[shakteebiswal]

@nguyenthai0107 Facing the same issue any idea on how to resolve?