bentoml / BentoML

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
https://bentoml.com
Apache License 2.0
7.15k stars 791 forks source link

bug: Error using bentoml.client.Client.from_url to create client #3634

Closed ryan-3m closed 1 year ago

ryan-3m commented 1 year ago

Describe the bug

I followed the documentation to create a BentoService that uses gRPC as its API server. The service can be called directly using gRPC without any issues. However, when I tried to create a client using bentoml.client.Client.from_url, I encountered an error.

I also tried to copy and paste the example from the documentation completely, but encountered the same error.

I encountered the following error:

Traceback (most recent call last):
  File "...\site-packages\IPython\core\interactiveshell.py", line 3460, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input>", line 2, in <module>
    c = bentoml.client.Client.from_url("localhost:3000")
  File "...\site-packages\bentoml\client.py", line 124, in from_url
    resp = conn.getresponse()
  File "...\lib\http\client.py", line 1377, in getresponse
    response.begin()
  File "...\lib\http\client.py", line 320, in begin
    version, status, reason = self._read_status()
  File "...\lib\http\client.py", line 302, in _read_status
    raise BadStatusLine(line)
http.client.BadStatusLine: 

My speculation is that the http module failed to decode the returned value, and the decoded string still looks like an encoded state.

To reproduce

Steps to reproduce:

  1. Create a BentoService that uses gRPC as its API server.
  2. Start the BentoService with the bentoml serve command.
  3. Try to create a client using bentoml.client.Client.from_url

with the following code:

Server:

import bentoml
import numpy as np

from bentoml.io import Image, NumpyNdarray

svc = bentoml.Service(
    name='sample_service',
)

@svc.api(
    input=NumpyNdarray(),
    output=NumpyNdarray()
)
async def some_service(image):
    return image

Client:

import bentoml

c = bentoml.client.Client.from_url("localhost:3000")

Expected behavior

The client should be created successfully and I should be able to use it to call the BentoService using gRPC.

Environment

Operating system: Windows/Linux Python version: 3.9/3.10 BentoML version: 1.0.15

aarnphm commented 1 year ago

This is not a bug. The client in 1.0.15 doesn't support gRPC. #3280 provides support for gRPC client and will be included in 1.0.16

ryan-3m commented 1 year ago

Thank you for your response. I realized that the documentation was not written specifically for version 1.0.15, and I did not notice that "latest" in the documentation does not necessarily correspond to version 1.0.15.

I will close this issue for now.

Thanks again for your help.

sauyon commented 1 year ago

Maybe it would be best for us to freeze docs on release, or at least hold features docs PRs until release so this doesn't happen :sweat_smile:.

aarnphm commented 1 year ago

Probably better to set the default docs to the latest release branch instead of main.