-
### System Info
GPU: Nvidia H100
Model: Llama3 8B
### Who can help?
@kaiyux
### Information
- [x] The official example scripts
- [ ] My own modified scripts
### Tasks
- [ ] An officially suppo…
-
### Prerequisites
- [X] I am running the latest code. Mention the version if possible as well.
- [X] I carefully followed the [README.md](https://github.com/ggerganov/llama.cpp/blob/master/README.md)…
-
### Proposal to improve performance
_No response_
### Report of performance regression
_No response_
### Misc discussion on performance
---
**Setup Summary for vLLM Benchmarking with Llama…
-
Config:
```
esphome:
name: lora32-home
friendly_name: Lora32_home
libraries:
- "SPI"
- "Ticker"
- "SX126x-Arduino"
esp32:
board: esp32-s3-devkitc-1
framework:
…
-
### System Info
a100
### Who can help?
@byshiue
@juney-nvidia
### Information
- [ ] The official example scripts
- [x] My own modified scripts
### Tasks
- [x] An officially supported task in th…
-
Usually you can use LoRA extraction in mergekit and then run the LoRAs in vLLM or Aphrodite Engine just fine. This works for Llama and Mistral models so far, but it seems like this isn't working for Q…
-
Updating /config/esphome/lorawan.yaml
------------------------------------------------------------
INFO ESPHome 2024.10.3
INFO Reading configuration /config/esphome/lorawan.yaml...
ERROR Unexpec…
-
### Your current environment
AMD radon + kubernetes
### Model Input Dumps
`vllm serve mistralai/Mistral-7B-Instruct-v0.3 --trust-remote-code --enable-chunked-prefill --max_num_batch…
-
It was working fine until the update. But now this.
Exception:
```
KeyError
Python 3.10.7: D:\Graphics\Krita (x64)\bin\krita.exe
Tue Nov 19 20:18:55 2024
A problem occurred in a Python scrip…
-
### Checklist
- [X] 1. I have searched related issues but cannot get the expected help.
- [X] 2. The bug has not been fixed in the latest version.
- [X] 3. Please note that if the bug-related issue y…