-
Open this issue for tracking the progress of models supported in candle-vllm.
-
### Your current environment
```Collecting environment information...
/app/apps/anaconda3/envs/vllm_053p1/lib/python3.9/site-packages/requests/__init__.py:102: RequestsDependencyWarning: urllib3 (1.…
-
### Your current environment
Device: Nvidia GeForce 4090
software: vllm 0.5.2 + openai 1.30.5 + transformes 4.42.4
### 🐛 Describe the bug
I use OpenAI api and vllm to deploy local Qwen2 ll…
-
## User Story: Implement Backend Prometheus Metrics
**As a** backends operator
**I want** to have Prometheus metrics for observability of the vLLM backend
**So that** I can monitor the performance, h…
-
-
### Your current environment
```text
Collecting environment information...
WARNING 07-22 09:16:28 _custom_ops.py:14] Failed to import from vllm._C with ModuleNotFoundError("No module named 'vllm._C…
-
### What would you like to see?
it would be great to be able to configure AnythingLLM with a Vllm model
https://github.com/vllm-project/vllm
-
yuanmodel/vllm-v0.4.0:latest
-
yuanmodel/vllm-v0.4.0:latest
-
https://github.com/opea-project/GenAIComps/tree/main/comps/llms#1-start-microservice-with-python-option-1
docker run -it --name vllm_service -p 8008:80 -e HF_TOKEN=${HUGGINGFACEHUB_API_TOKEN} -v ./…