-
This RFC proposes improvements to the management of Low-Rank Adaptation (LoRA) in vLLM to make it more suitable for production environments. This proposal aims to address several pain points observed …
-
### Checklist
- [ ] 1. I have searched related issues but cannot get the expected help.
- [ ] 2. The bug has not been fixed in the latest version.
- [ ] 3. Please note that if the bug-related issue y…
-
### Expected Behavior
The UI Is usable
### Actual Behavior
The UI runs at extremely low framerates and frequently locks up, sometimes to the point that the entire tab dies (Chrome, Degoogled…
-
### Anything you want to discuss about vllm.
I've fine-tuned Qwen2.5-14B-Instruct using QLora(bitsandbytes 4bit) and also a full fine-tune. However when I tried to use it with a quantized model (Qw…
-
I trained a LoRA of Qwen2.5-Coder-7B-Instruct using ms-swift, and merge it with ms-swift.
The Command and the output of terminal is below.
Train, merge and app-ui command
swift sft \
--mod…
-
-
### System Info
2X L4 GPUs
Docker Image:
nvcr.io/nvidia/tritonserver:24.06-trtllm-python-py3
### Who can help?
@juney-nvidia @kaiyux
### Information
- [ ] The official example sc…
-
### Related area
Heltec LoRa ESP32 V2
### Hardware specification
esp32-s3
### Is your feature request related to a problem?
Hello,
I am experiencing difficulties connecting my Heltec LoRa ESP…
-
Here is the development roadmap for 2024 Q4. Contributions and feedback are welcome ([**Join Bi-weekly Development Meeting**](https://t.co/4BFjCLnVHq)). Previous 2024 Q3 roadmap can be found in #634.
…
-
### Checklist
- [x] The issue exists after disabling all extensions
- [ ] The issue exists on a clean installation of webui
- [ ] The issue is caused by an extension, but I believe it is caused b…