-
Hello, may I ask if I can run the model by adjusting H and W with only V100 32GB, especially when using multiple GPUs?
-
### What happened + What you expected to happen
I'm seeing multiple issues (all related to matrix dimensions it seems) for all multivariate models (Except HINT because I could not determine S param…
-
**Is this a BUG REPORT or FEATURE REQUEST?**: feature request
/kind feature
**What happened**:
As far, we do not support sharing GPU to multiple containers, one GPU can only be assigned to…
-
First of all, thank you for maintaining the code. I would like to try using my own dataset and training on multiple GPUs, but the following issues may arise. Very strange.
-------------------------…
-
I'm using lm-eval v0.4.2 to evaluate Llama 7b on the open llm leaderboard benchmark.
I found that there are accuracy gaps between single GPU and multiple GPUs as below. (I used data parallel)
| |…
-
I found with 2d5-7b the checkpoint saved from LoRA tuning finetune.py with one GPU is correct, while with multiple GPU the model saved is incorrect.
Does anyone met similar problem?
For example…
-
Hi, I'm currently trying to run DeepSeek Coder v2 on a single node with the following setup:
Node 1: Two A6000 GPUs (48GB each) and 192GB of RAM
Node 2: Two 4090 GPUs (24GB each) and 64GB …
-
Fastchat to enable baichuan2 LLM to use openai invoke, two V100 32G GPUs. It is more slower than model running with one GPU when inference. Nearly 3 tokens in 5 seconds.
```
python3 -m fastchat.serv…
-
Is it possible to build "fat binary" which is compatible with multiple GPU architectures?
I'm able to build binary with OpenMP target offload for single GPU architecture (such as AOMP_GPU=gfx900), b…
-
We tried to launch the model worker on a machine with multiple RTX3090 GPUs, but we couldn't use this command `python -m llava.serve.model_worker --host 0.0.0.0 --controller http://localhost:10000 --p…