Closed todtjs92 closed 1 week ago
Is it not possible to use grpc-v2 in kserve-lgbserver?
there are only v1 and v2 on this page: https://github.com/kserve/kserve/blob/master/config/runtimes/kserve-lgbserver.yaml
@todtjs92 You should pass the argument --grpc_port=9000
to the model server to change the port otherwise grpc server will listen on 8081
@todtjs92 Ensure that your service is correctly routing the traffic to the right endpoints. The 404 error indicates that the endpoint might not be found or correctly configured. Ensure that the gRPC server is correctly configured to run on the expected port. By default, it seems to start on port 8081. Update the command in your Dockerfile and InferenceService YAML to ensure that the server is listening on the correct port.
@sivanantha321 @LOADBC Thanks all!! It seems like i set the wrong port for grpc... .I'll try to fix it. Thank you so much!..
/kind bug
What steps did you take and what happened:
iam trying to practice transformer example below: https://kserve.github.io/website/latest/modelserving/v1beta1/transformer/torchserve_image_transformer/#create-inferenceservice
I only change the model pytorch to lgbm and it works when i use v1 + rest protocol. But when i tried to practice {Deploy the InferenceService calling Predictor with gRPC protocol} example , error is occured. Here are my files .
InferenceService
transformer.py
dockerfile
Error
error is too long the point is here .
and the weird thing is i set the port 9000 but the log said it start grpc server on port 8081
All log
What did you expect to happen: request must be success
What's the InferenceService yaml: [To help us debug please run
kubectl get isvc $name -n $namespace -oyaml
and paste the output] Above!Anything else you would like to add: [Miscellaneous information that will assist in solving the issue.] Another weird thing is
Environment:
kubectl version
):/etc/os-release
):