-
### Your current environment
```text
Collecting environment information...
PyTorch version: 2.3.1+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
…
-
### Your current environment
```text
PyTorch version: 2.3.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
OS: Ubuntu 22.04.4 LTS (x86_64)
GCC ve…
-
### Your current environment
PyTorch version: 2.4.1+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
OS: Ubuntu 22.04.3 LTS (x86_64)
GCC version: (U…
-
I used the Chinese manual document to build it, and found that the extracted entities were very messy. Is there any good way to optimize it?
-
### What is the issue?
This is on AMD. I have 2 x Radeon 7900 XCX cards (24gb each).
For models/memory use that only uses 1 GPU, everything works fine.
As soon as both cards are required, the inf…
-
### What is the issue?
Hello everyone, thank you for your work.
I'm using a Jetson Nano Orin. Following #3098, some days ago I done a ```git checkout``` using #2279 commit and install this versi…
-
### What happened?
The quantized version of gemma 27b (Q8_0) still gets the answer wrong to even simple problems.
The version of gemma on ai studio answers correctly all my questions.
Example pro…
-
I'm trying to follow the instructions for installing unsloth in a conda environment, the problem is that the conda gets stuck when running the install lines.
I've tried running it twice, both times…
-
I have noticed that Alpaca uses my CPU instead of my GPU. Here's a screenshot showing how it's using almost 40% of my CPU, and only 1% of my GPU.
![Captura desde 2024-07-10 06-51-39](https://github…
-
### Your current environment
Not applicable -- Dockerfile.
### 🐛 Describe the bug
Steps to reproduce:
- Clone the `vllm` repo
- run `docker build . --target vllm-base`
- Build fails
```shel…