-
### Is this a bug or a feature request?
bug
### What did you expect?
build lora-app-server on Raspberry pi 3 , running alpine arm64
### What happened?
```
Generating combined Swagger JSON
[…
-
### Your current environment
```text
The output of `python collect_env.py`
```
### 🐛 Describe the bug
I'm running the latest docker image and an openai style endpoint.
My command is:
```
-…
-
### How would you like to use vllm
I want to run inference of a [TheBloke/Llama-2-7B-Chat-GPTQ](https://huggingface.co/TheBloke/Llama-2-7B-Chat-GPTQ). I don't know how to use it with vllm.
I try t…
-
### Expected Behavior
-
### Actual Behavior
![image](https://github.com/user-attachments/assets/1f9608dc-4631-41c3-bd2a-bfe506d39104)
SD15 and Flux work fine, the problem is only with SDXL
Co…
-
### System Info / 系統信息
操作系统=ubuntu20.04
显卡=4 x v100
model=glm-9b-chat
python=3.11.8
llama-factory=0.8.3
transformers=4.43.3
vllm=0.5.3.post1
### Who can help? / 谁可以帮助到您?
_No response_
### …
-
### Is there an existing issue for this?
- [X] I have searched the existing issues and checked the recent builds/commits
### What happened?
I started 3 WebUI servers on one Nvidia GPU L40, the gen…
-
Hi all,
I want to integrate the sx1302 lorawan packet forwarder with aws server it requires aws key, certifates, trust certificates.
I am trying to configure with sample configuration files am not…
-
**Is your feature request related to a problem? Please describe.**
This is a feature related to how to deploy a model with LoRA supported.
**Describe the solution you'd like**
I have a UNet model…
-
- --gpus should be a top level option when serving a model since you can start a server separately from most cmds.
- vllm-args should be exposed when running the serve command
- passing additonal a…
-
I need to change the variable that I received by LoRa for to send to ThingsBoard, but there is a error. "invalid conversion from 'int' to 'const char*' [-fpermissive]"
Thank you!
```#include
#…