search

bentoml / BentoML

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
https://bentoml.com
Apache License 2.0
7.16k stars 792 forks source link

bug: SSL does not work #4710

Closed ProVega closed 5 months ago

ProVega commented 6 months ago

Describe the bug

Configure SSL per instructions here - https://docs.bentoml.com/en/latest/guides/configurations.html

ssl={ "enabled": True, "certfile": "/fullpath/file.cer", "keyfile": "/fullpath/file.key", "keyfile_password": "password"

Using the exact same config into Uvicorn directly, with same key and cert and password works just fine.

I don't see how to turn on additional logging, so I don't really know what the problem is.

All requests, get this warning. 2024-05-06T07:53:05-0600 [WARNING] [entry_service:bentovllm-llama3-8b-insruct-service:1] Invalid HTTP request received. 2024-05-06T07:53:05-0600 [WARNING] [entry_service:bentovllm-llama3-8b-insruct-service:1] Invalid HTTP request received.

Enabling DEBUG doesn't yield any additional info (yes I see [DEBUG] in the output for other stuff)

To reproduce

No response

Expected behavior

SSL should work

Environment

bentoml: 1.2.12 uvicorn: 0.9.0

ProVega commented 5 months ago

Bump - Does anyone have any guidance on how to get SSL working?

ProVega commented 5 months ago

Wow, after upgrading from 1.2.13 to 1.2.18 the issue magically has gone away. Same config, same cert.