-
does spring ai support the vllm +qwen?
which starter can i use? and how to use spring ai connect the vllm ?
tks.
-
Hello,
While using the ELI5 and TriviaQA datasets from the Hugging Face library, I encountered errors related to missing documents that are not present in the corpus. I experienced a similar issue …
gnekt updated
2 weeks ago
-
### Enhancement Description
Update the vLLM version from 0.3.2 to the latest available version. This update is necessary to support the phi-3 mini model, which is only compatible with vLLM 0.4.3 and …
-
Hi, using version 0.10.3 and the llama3 tokenizer, with vLLM, I can't seem to constrain to generate emojis.
```
curl --request POST \
--url http://localhost:8000/v1/chat/completions \
--hea…
-
INFO: 172.16.80.35:48532 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/home/bigdata/anacon…
-
noob here, does this mean no mac support, "AssertionError: vLLM only supports Linux platform (including WSL)."
-
With mutiple gpus configured, vllm serving is only shutting down 1 of the workers on shutdown
**To Reproduce**
Steps to reproduce the behavior:
1. ilab serve --model-path models/instructlab/gra…
-
Hi, when I follow the default steps to set up environment:
pip install vllm
it will automaticly install vllm 0.5.0.post1, and transformers>=4.40.0 is required.
When installing SPPO ( transformer…
-
I wonder is quanto-quantized model available using vllm?
-
I'm trying to run/load prometheus on Amazon Sagemaker Stuidio notebooks but keep running into errors.
If I load it using VLLM
`model = VLLM(model="prometheus-eval/prometheus-7b-v2.0")`
`ValueErro…