bentoml / BentoML

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
https://bentoml.com
Apache License 2.0
7.13k stars 791 forks source link

An easier way to include a bento_server configuration file inside of the bento #2312

Closed timliubentoml closed 2 years ago

timliubentoml commented 2 years ago

Is your feature request related to a problem? Please describe. Currently the bento server can be configured using a configuration file as described here: https://docs.bentoml.org/en/latest/guides/configuration.html

However, the only way to use it with the containerized bento is by including the configuration file in the bento, then specifying where it is when calling docker run, either by setting an environmental variable, or by specifying --config which is passed to the docker serve command.

Alternatively you could modify the Dockerfile or the entrypoint.sh to set the environment variable, however this will get overridden each time you create a new bento

Describe the solution you'd like Would like a parameter in the bentofile.yaml which could point to a specific configuration file and then it is included in the bento as well as configured to point to it in the container.

Describe alternatives you've considered Could potentially merge bentoserver configuration with bentofile.yaml? though that makes local dev a little harder.

Open to other suggestions. Generally it would be nice to have a more easy way to package the bento server configuration into the docker container without having to tweak out it's run. Sometimes people don't have the ability to add parameters to the end of the docker run command because they are using automatic deployments which don't easily allow for customization

Additional context https://bentoml.slack.com/archives/CKRANBHPH/p1645522385513969

ssheng commented 2 years ago

An option is to add a configuration section in bentofile.yml.

ssheng commented 2 years ago