The difference in bentoml serve and bentoml serve-grpc
This only seems to be the case for class based services. "variable" based ones seem to work just fine.
Log output
```
2024-04-05T16:15:51+0200 [WARNING] [cli] Converting 'BentoService' to lowercase: 'bentoservice'.
2024-04-05T16:15:51+0200 [DEBUG] [cli] Failed to load bento with v1.2 loader, fallback to old loader
2024-04-05T16:15:51+0200 [DEBUG] [cli] Importing service "service:BentoService" from working dir: "C:\Users\User\source\repos\bento-service\src"
2024-04-05T16:15:52+0200 [DGB] [cli] 'bento_service' imported from source:
2024-04-05T16:15:53+0200 [DGB] [cli] Tracking Error: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1006)
Error: [serve] `serve-grpc` failed: type doesn't support gRPC serving
Traceback (most recent call last):
File "C:\Users\User\mambaforge\envs\bentoml\Scripts\bentoml-script.py", line 9, in
sys.exit(cli())
^^^^^
File "C:\Users\User\mambaforge\envs\bentoml\Lib\site-packages\click\core.py", line 1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\User\mambaforge\envs\bentoml\Lib\site-packages\click\core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "C:\Users\User\mambaforge\envs\bentoml\Lib\site-packages\click\core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\User\mambaforge\envs\bentoml\Lib\site-packages\click\core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\User\mambaforge\envs\bentoml\Lib\site-packages\click\core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\User\mambaforge\envs\bentoml\Lib\site-packages\bentoml_cli\utils.py", line 385, in wrapper
raise err from None
File "C:\Users\User\mambaforge\envs\bentoml\Lib\site-packages\bentoml_cli\utils.py", line 380, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\User\mambaforge\envs\bentoml\Lib\site-packages\click\decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\User\mambaforge\envs\bentoml\Lib\site-packages\bentoml_cli\utils.py", line 351, in wrapper
return_value = func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\User\mambaforge\envs\bentoml\Lib\site-packages\bentoml_cli\env_manager.py", line 122, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\User\mambaforge\envs\bentoml\Lib\site-packages\bentoml_cli\serve.py", line 500, in serve_grpc
serve_grpc_production(
File "C:\Users\User\mambaforge\envs\bentoml\Lib\site-packages\simple_di\__init__.py", line 139, in _
return func(*_inject_args(bind.args), **_inject_kwargs(bind.kwargs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\User\mambaforge\envs\bentoml\Lib\site-packages\bentoml\serve.py", line 552, in serve_grpc_production
raise BentoMLException(f"{type(svc)} type doesn't support gRPC serving")
bentoml.exceptions.BentoMLException: type doesn't support gRPC serving
```
To reproduce
from __future__ import annotations
import logging
import numpy as np
import bentoml
import numpy as np
root_logger = logging.getLogger()
root_logger.setLevel(logging.INFO)
bentoml_logger = logging.getLogger("bentoml")
bentoml_logger.setLevel(logging.DEBUG)
@bentoml.service(name="bento_service")
class BentoService():
def __init__(self) -> None:
pass
@bentoml.api #(input=bentoml.io.NumpyNdarray(np.float64,shape=[1,1]), output=bentoml.io.NumpyNdarray(np.float64,shape=[1,1]))
async def double_it(self, input: np.ndarray) -> np.ndarray:
result = input * 2
return result
and then run this:
bentoml serve-grpc --development --verbose
Expected behavior
This should work just like the http serve, and actually start the service.
Environment
bentoml: 1.2.9
python: 3.11.8
platform: Windows 11 x64 23H2
Describe the bug
The difference in
bentoml serve
andbentoml serve-grpc
This only seems to be the case for class based services. "variable" based ones seem to work just fine.
Log output
``` 2024-04-05T16:15:51+0200 [WARNING] [cli] Converting 'BentoService' to lowercase: 'bentoservice'. 2024-04-05T16:15:51+0200 [DEBUG] [cli] Failed to load bento with v1.2 loader, fallback to old loader 2024-04-05T16:15:51+0200 [DEBUG] [cli] Importing service "service:BentoService" from working dir: "C:\Users\User\source\repos\bento-service\src" 2024-04-05T16:15:52+0200 [DGB] [cli] 'bento_service' imported from source:To reproduce
and then run this:
bentoml serve-grpc --development --verbose
Expected behavior
This should work just like the http serve, and actually start the service.
Environment
bentoml: 1.2.9 python: 3.11.8 platform: Windows 11 x64 23H2