bentoml / BentoML

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and much more!
https://bentoml.com
Apache License 2.0
7.12k stars 791 forks source link

bug: Working Service for HTTP gives error with Grpc "type doesn't support gRPC serving" #4634

Open EraYaN opened 7 months ago

EraYaN commented 7 months ago

Describe the bug

The difference in bentoml serve and bentoml serve-grpc

This only seems to be the case for class based services. "variable" based ones seem to work just fine.

Log output ``` 2024-04-05T16:15:51+0200 [WARNING] [cli] Converting 'BentoService' to lowercase: 'bentoservice'. 2024-04-05T16:15:51+0200 [DEBUG] [cli] Failed to load bento with v1.2 loader, fallback to old loader 2024-04-05T16:15:51+0200 [DEBUG] [cli] Importing service "service:BentoService" from working dir: "C:\Users\User\source\repos\bento-service\src" 2024-04-05T16:15:52+0200 [DGB] [cli] 'bento_service' imported from source: 2024-04-05T16:15:53+0200 [DGB] [cli] Tracking Error: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1006) Error: [serve] `serve-grpc` failed: type doesn't support gRPC serving Traceback (most recent call last): File "C:\Users\User\mambaforge\envs\bentoml\Scripts\bentoml-script.py", line 9, in sys.exit(cli()) ^^^^^ File "C:\Users\User\mambaforge\envs\bentoml\Lib\site-packages\click\core.py", line 1157, in __call__ return self.main(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\User\mambaforge\envs\bentoml\Lib\site-packages\click\core.py", line 1078, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File "C:\Users\User\mambaforge\envs\bentoml\Lib\site-packages\click\core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\User\mambaforge\envs\bentoml\Lib\site-packages\click\core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\User\mambaforge\envs\bentoml\Lib\site-packages\click\core.py", line 783, in invoke return __callback(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\User\mambaforge\envs\bentoml\Lib\site-packages\bentoml_cli\utils.py", line 385, in wrapper raise err from None File "C:\Users\User\mambaforge\envs\bentoml\Lib\site-packages\bentoml_cli\utils.py", line 380, in wrapper return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\User\mambaforge\envs\bentoml\Lib\site-packages\click\decorators.py", line 33, in new_func return f(get_current_context(), *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\User\mambaforge\envs\bentoml\Lib\site-packages\bentoml_cli\utils.py", line 351, in wrapper return_value = func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\User\mambaforge\envs\bentoml\Lib\site-packages\bentoml_cli\env_manager.py", line 122, in wrapper return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\User\mambaforge\envs\bentoml\Lib\site-packages\bentoml_cli\serve.py", line 500, in serve_grpc serve_grpc_production( File "C:\Users\User\mambaforge\envs\bentoml\Lib\site-packages\simple_di\__init__.py", line 139, in _ return func(*_inject_args(bind.args), **_inject_kwargs(bind.kwargs)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\User\mambaforge\envs\bentoml\Lib\site-packages\bentoml\serve.py", line 552, in serve_grpc_production raise BentoMLException(f"{type(svc)} type doesn't support gRPC serving") bentoml.exceptions.BentoMLException: type doesn't support gRPC serving ```

To reproduce

from __future__ import annotations

import logging

import numpy as np

import bentoml
import numpy as np

root_logger = logging.getLogger()
root_logger.setLevel(logging.INFO)

bentoml_logger = logging.getLogger("bentoml")
bentoml_logger.setLevel(logging.DEBUG)

@bentoml.service(name="bento_service")
class BentoService():

    def __init__(self) -> None:
       pass

    @bentoml.api #(input=bentoml.io.NumpyNdarray(np.float64,shape=[1,1]), output=bentoml.io.NumpyNdarray(np.float64,shape=[1,1]))
    async def double_it(self, input: np.ndarray) -> np.ndarray:
        result = input * 2
        return result

and then run this: bentoml serve-grpc --development --verbose

Expected behavior

This should work just like the http serve, and actually start the service.

Environment

bentoml: 1.2.9 python: 3.11.8 platform: Windows 11 x64 23H2

xianml commented 5 months ago

Hi. thanks for reporting. Bentoml 1.2 Service have not impl Grpc support yet.