-
**Is your feature request related to a problem? Please describe.**
The new ImageInput with micro-batching support does not allow multiple image inputs, comparing to the old ImageHandler
**De…
-
### Describe the bug
Everytime i am getting this error when I am running these cmd-
- bentoml build
- bentoml list
![image](https://github.com/bentoml/BentoML/assets/68845761/8ac12da4-…
-
### System Information
ZENML_LOCAL_VERSION: 0.57.1
ZENML_SERVER_VERSION: 0.57.1
ZENML_SERVER_DATABASE: sqlite
ZENML_SERVER_DEPLOYMENT_TYPE: other
ZENML_CONFIG_DIR: /root/.config/zenml
ZE…
-
### Motivation
As we know, the current performance of LMDeploy PyTorch Engine **slightly outperforms** vLLM in throughput. Currently, vLLM is being refactored and aims to integrate with FlashInfer. […
-
I've tried to run zephyr-7b-alpha in docker container via Docker Desktop on Windows 10. I get the following error, when the server starts:
> [ERROR] [runner:llm-mistral-runner:1] An exception occur…
cay89 updated
7 months ago
-
I am opening this issue to discuss and report on our progress towards removing hard requirement for BentoML in Ersilia. While this library has been very useful thus far, it is causing dependency issue…
-
### Describe the bug
I'm running dind in Gitlab CI runner and the bentoml containerize command breaks with the following error:
docker version is: Docker version 25.0.3, build 4debf41
I have `DOCKE…
-
### Describe the bug
this configuration is not working
```
version: 1 # should be same to the BentoML major version
api_server:
tracing:
exporter_type: zipkin
sample_rate: 1.0
…
-
### Brief summary
I'm doing some load testing on a `BentoML` server in Python that runs an image classification model. Under large numbers of virtual users, I get back lots of EOF and status code 0 e…
-
### System Info
transformers-4.37.0.dev0
### Who can help?
@Narsil
### Information
- [X] The official example scripts
- [ ] My own modified scripts
### Tasks
- [X] An officially…