search

kserve / website

User documentation for KServe.
https://kserve.github.io/website/
Apache License 2.0
103 stars 109 forks source link

Unable to run inference service #355

Closed VikasAbhishek closed 1 month ago

VikasAbhishek commented 3 months ago

I have followed the official documentation and deployed the serverless deployment, but after running the first inference service example. I am unable to get desired result that has shown in the page. Please find attached image here with

I followed this link for running the inference service https://kserve.github.io/website/master/get_started/first_isvc/#5-perform-inference

Screenshot 2024-04-01 185643
spolti commented 3 months ago

Hi @VikasAbhishek, what the command below returns?

kubectl get isvc
VikasAbhishek commented 3 months ago

Classification: Confidential Hi Filippe, Please find attached image.

From: Filippe Spolti @.> Sent: Monday, April 1, 2024 7:40 PM To: kserve/website @.> Cc: Vikas Ghunawat Meena @.>; Mention @.> Subject: Re: [kserve/website] Unable to run inference service (Issue #355)

[CAUTION: This Email is from outside the Organization. Unless you trust the sender, Don't click links or open attachments as it may be a Phishing email, which can steal your Information and compromise your Computer.]

Hi @VikasAbhishekhttps://github.com/VikasAbhishek, what the command below returns?

kubectl get isvc

- Reply to this email directly, view it on GitHubhttps://github.com/kserve/website/issues/355#issuecomment-2029817867, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BHL3AUQVDG6GF26CPMYY6I3Y3FTDDAVCNFSM6AAAAABFRTVHB2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRZHAYTOOBWG4. You are receiving this because you were mentioned.Message ID: @.**@.>>

::DISCLAIMER::

The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.

spolti commented 3 months ago

there is nothing attached, just pasting the text here should be enough,.

VikasAbhishek commented 3 months ago

Classification: Confidential @.***:/home/ubuntu# kubectl get isvc -n kserve-test NAME URL READY PREV LATEST PREVROLLEDOUTREVISION LATESTREADYREVISION AGE sklearn-iris http://sklearn-iris.kserve-test.svc.cluster.localhttp://sklearn-iris.kserve-test.svc.cluster.local/ True 100 sklearn-iris-predictor-00003 5d6h

From: Filippe Spolti @.> Sent: Tuesday, April 2, 2024 7:07 PM To: kserve/website @.> Cc: Vikas Ghunawat Meena @.>; Mention @.> Subject: Re: [kserve/website] Unable to run inference service (Issue #355)

[CAUTION: This Email is from outside the Organization. Unless you trust the sender, Don't click links or open attachments as it may be a Phishing email, which can steal your Information and compromise your Computer.]

there is nothing attached, just pasting the text here should be enough,.

- Reply to this email directly, view it on GitHubhttps://github.com/kserve/website/issues/355#issuecomment-2032068195, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BHL3AUXHNDP4WMWROT6GBW3Y3KYBLAVCNFSM6AAAAABFRTVHB2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZSGA3DQMJZGU. You are receiving this because you were mentioned.Message ID: @.**@.>>

::DISCLAIMER::

The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.

sivanantha321 commented 3 months ago

@VikasAbhishek Seems like you have not configured a domain or DNS. If you try to infer from outside the cluster it won't work

VikasAbhishek commented 3 months ago

Classification: Confidential Hi Sivanantham, Can you please guide me how to configure DNS and in which services should I make the changes. If you have any example file for the same, please share with me.

Thank you, Vikas Ghunawat Meena

From: Sivanantham @.> Sent: Wednesday, April 3, 2024 12:34 AM To: kserve/website @.> Cc: Vikas Ghunawat Meena @.>; Mention @.> Subject: Re: [kserve/website] Unable to run inference service (Issue #355)

[CAUTION: This Email is from outside the Organization. Unless you trust the sender, Don't click links or open attachments as it may be a Phishing email, which can steal your Information and compromise your Computer.]

@VikasAbhishekhttps://github.com/VikasAbhishek Seems like you have not configured a domain or DNS. If you try to infer from outside the cluster it won't work

- Reply to this email directly, view it on GitHubhttps://github.com/kserve/website/issues/355#issuecomment-2032844184, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BHL3AUQNHV5IZQ5LZM6WCC3Y3L6JTAVCNFSM6AAAAABFRTVHB2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZSHA2DIMJYGQ. You are receiving this because you were mentioned.Message ID: @.**@.>>

::DISCLAIMER::

The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.

sivanantha321 commented 3 months ago

Here is the docs from knative for configuring domain. If you are just doing it for testing purposes, then you can just use this command below and you can use the kubectl port-forward to access the cluster without the need to configuring DNS.

kubectl patch cm config-domain --patch '{"data":{"example.com":""}}' -n knative-serving

If you want to configure DNS follow this doc

VikasAbhishek commented 3 months ago

Classification: Confidential @.***:/home/ubuntu# kubectl logs load-testf4ps9-qznqs -n kserve-test Requests [total, rate, throughput] 30000, 500.02, 0.00 Duration [total, attack, wait] 59.999s, 59.998s, 663.663µs Latencies [min, mean, 50, 90, 95, 99, max] 427.534µs, 790.51µs, 748.113µs, 955.089µs, 1.052ms, 1.593ms, 26.421ms Bytes In [total, mean] 0, 0.00 Bytes Out [total, mean] 2460000, 82.00 Success [ratio] 0.00% Status Codes [code:count] 404:30000 Error Set: 404 Not Found

I am getting these logs while running the performance test.

From: Sivanantham @.> Sent: Thursday, April 4, 2024 12:22 PM To: kserve/website @.> Cc: Vikas Ghunawat Meena @.>; Mention @.> Subject: Re: [kserve/website] Unable to run inference service (Issue #355)

[CAUTION: This Email is from outside the Organization. Unless you trust the sender, Don't click links or open attachments as it may be a Phishing email, which can steal your Information and compromise your Computer.]

Here is the docs from knative for configuring domainhttps://knative.dev/docs/serving/using-a-custom-domain/. If you are just doing it for testing purposes, then you can just use this command below and you can use the kubectl port-forward to access the cluster without the need to configuring DNS.

kubectl patch cm config-domain --patch '{"data":{"example.com":""}}' -n knative-serving

If you want to configure DNS follow this dochttps://knative.dev/docs/install/yaml-install/serving/install-serving-with-yaml/#configure-dns

- Reply to this email directly, view it on GitHubhttps://github.com/kserve/website/issues/355#issuecomment-2036326725, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BHL3AUUXD2RMECSLURDQVV3Y3T2BBAVCNFSM6AAAAABFRTVHB2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZWGMZDMNZSGU. You are receiving this because you were mentioned.Message ID: @.**@.>>

::DISCLAIMER::

The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.

VikasAbhishek commented 3 months ago

Classification: Confidential @.***:/home/ubuntu# curl -v -H "Host: ${SERVICE_HOSTNAME}" -H "Content-Type: application/json" http://${INGRESS_HOST}:${INGRESS_PORT}/v1/models/sklearn-iris:predicthttp://$%7bINGRESS_HOST%7d:$%7bINGRESS_PORT%7d/v1/models/sklearn-iris:predict -d @./iris-input.json

I am still getting the same issue.

From: Sivanantham @.> Sent: Thursday, April 4, 2024 12:22 PM To: kserve/website @.> Cc: Vikas Ghunawat Meena @.>; Mention @.> Subject: Re: [kserve/website] Unable to run inference service (Issue #355)

[CAUTION: This Email is from outside the Organization. Unless you trust the sender, Don't click links or open attachments as it may be a Phishing email, which can steal your Information and compromise your Computer.]

Here is the docs from knative for configuring domainhttps://knative.dev/docs/serving/using-a-custom-domain/. If you are just doing it for testing purposes, then you can just use this command below and you can use the kubectl port-forward to access the cluster without the need to configuring DNS.

kubectl patch cm config-domain --patch '{"data":{"example.com":""}}' -n knative-serving

If you want to configure DNS follow this dochttps://knative.dev/docs/install/yaml-install/serving/install-serving-with-yaml/#configure-dns

- Reply to this email directly, view it on GitHubhttps://github.com/kserve/website/issues/355#issuecomment-2036326725, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BHL3AUUXD2RMECSLURDQVV3Y3T2BBAVCNFSM6AAAAABFRTVHB2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZWGMZDMNZSGU. You are receiving this because you were mentioned.Message ID: @.**@.>>

::DISCLAIMER::

The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.

Syntax-Error-1337 commented 2 months ago

@VikasAbhishek

VikasAbhishek commented 1 month ago

Resolved.