search

bentoml / BentoML

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
https://bentoml.com
Apache License 2.0
7.17k stars 792 forks source link

The YataiService gRPC server doesn't support URL location prefix #1329

Closed withsmilo closed 3 years ago

withsmilo commented 3 years ago

Describe the bug

When I run YataiService with --web-prefix-path option, the YataiService gRPC server doesn't have a prefix path. I think https://github.com/bentoml/BentoML/issues/1063 missed this bug.

To Reproduce

Expected behavior

The YataiService gRPC server should be executed on 127.0.0.1:50051/test_prefix.

Screenshots/Logs

BENTOML__CORE__DEBUG=true bentoml yatai-service-start --web-prefix-path test_prefix --verbose
[2020-12-16 09:57:52,224] DEBUG - Setting debug mode: ON for current session
[2020-12-16 09:57:52,367] DEBUG - Upgrading tables to the latest revision
[2020-12-16 09:57:52,374] DEBUG - Enabling gRPC server reflection for debugging
* Starting BentoML YataiService gRPC Server
* Debug mode: on
* Web UI: running on http://127.0.0.1:3000/test_prefix
* Running on 127.0.0.1:50051 (Press CTRL+C to quit)
* Usage:
*  Set config: `bentoml config set yatai_service.url=127.0.0.1:50051`
*  Set env var: `export BENTOML__YATAI_SERVICE__URL=127.0.0.1:50051`
* Help and instructions: https://docs.bentoml.org/en/latest/guides/yatai_service.html
* Web server log can be found here: /Users/user/bentoml/logs/yatai_web_server.log

Environment:

Additional context

yubozhao commented 3 years ago

I think that's the expected behavior for the gRPC endpoints right now. Did you run into a use-case that you want to have the prefix for the gRPC endpoints?

withsmilo commented 3 years ago

@yubozhao Yes. I'd like to run the YataiService behind a reverse proxy server. Is it impossible?

yubozhao commented 3 years ago

It is definitely possible. That PR only added the support for the prefix on web UI. We will need to add the prefix support for the gRPC endpoints.

Is this a blocking issue for you right now?

withsmilo commented 3 years ago

@yubozhao It is not a blocking issue for me, but I think it is a necessary function for security.

yubozhao commented 3 years ago

Yes, that makes sense. I will add this to the roadmap and we will try to get this address in the next few releases

withsmilo commented 3 years ago

Thanks, @yubozhao !

parano commented 3 years ago

@withsmilo I thought the URL prefix was only meant for the browser when deploying a web server behind a proxy, gRPC should just work behind a reverse proxy without problem.

withsmilo commented 3 years ago

@parano You're right. It works without any problem, but there is a security problem that exposes gRPC server's port to users.

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.