search

bentoml / OpenLLM

Run any open-source LLMs, such as Llama, Mistral, as OpenAI compatible API endpoint in the cloud.
https://bentoml.com
Apache License 2.0
10.12k stars 641 forks source link

bug: `openllm.client` doesn't respect per request configuration #11

Closed aarnphm closed 1 year ago

aarnphm commented 1 year ago

Describe the bug

when I do

client.query("What is 3+1?", return_full_text=True)

for a running dolly server, it doesn't process return_full_text correctly

To reproduce

No response

Logs

No response

Environment

No response

aarnphm commented 1 year ago

Fixed on main