-
### Describe the bug
environment: Python 10
usage:
openllm start NousResearch/llama-2-13b-chat-hf
llm = OpenLLMAPI(address="http://some_address:3000/")
llm.complete("What are some hazards cru…
-
### Describe the bug
Brand new installation, ran:
openllm start facebook/opt-1.3b
Will produce the error:
It is recommended to specify the backend explicitly. Cascading backend might lead to…
-
### Your current environment
The output of `python collect_env.py`
```text
Collecting environment information...
PyTorch version: 2.4.0+cu121
Is debug build: False
CUDA used to build PyTorch…
larme updated
2 months ago
-
### Describe the bug
When performing inference of microsoft/phi-2, it hangs for 5 min and then throws 500
### To reproduce
1. `TRUST_REMOTE_CODE=True DTYPE=float32 openllm start microsoft/phi-2`
2…
-
### Describe the bug
In "bentofile.yaml", we only include "*.py" file and some text files, but it takes a too long time to build bento.
It takes more than 1 hour for building bentos, and the creat…
-
### Describe the bug
The following `docker run` command starts the service, shows the available models but fails to run the completion:
```
docker run --rm -it -p 3000:3000 ghcr.io/bentoml/open…
-
### Describe the bug
(codellama) amardeep.yadav@fintricity.com@codellamamachine:~$ pip install "openllm[vllm]"
Requirement already satisfied: openllm[vllm] in ./miniconda3/envs/codellama/lib/python3…
-
### Describe the bug
Configure SSL per instructions here -
https://docs.bentoml.com/en/latest/guides/configurations.html
ssl={
"enabled": True,
"certfile": "/fullpath/file.cer",
"keyfile": "/…
-
### Describe the bug
I am going to serve the Detectron2 model using BentoML.
Also, I want to image through bentoml containerize function.
However, an error occurs during containerize when install…
-
### Describe the bug
Hello OpenLLM Team,
I recently encountered an issue while attempting to run a Docker container with the BentoML OpenLLM package to host the Microsoft Phi-2 language model. Dur…