-
-
I can't seem to use multiple gpus
There are two gpus in the setup, but training seems to be done on only one gpu causing out of memory
How can this situation be resolved?
-
I ran into OOM error to train a huge data. I have 4 GPUs, and I wonder if there's a way to use multiple gpus?
-
I encountered the issue shown in the picture during training. Is it because of insufficient memory? How do I train using multiple GPUs?
我在训练时出现了如图所示的问题,是不是因为内存不足了,如何使用多卡训练
![image](https://github.co…
-
Hi,
I am trying to run inference with `llama2+13b` and I have 4 RTX3090 each with 24GB Memory, however I noticed that when I use the sample inference code, it only uses one GPU which causes out of …
-
### Problem Description
I've built the VLLM with ROCM docker from the repo and is working fine with the offline_inference.py code provided in the examples folder using single GPU. When I try to run t…
-
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Is your issue described in the documentation?
- [X] I have read the documentation
### Is your issue p…
-
Hi, I tried to run a python code that would use multiple GPUs. It is a Pytorch inference model that loads on a Gpu to classify an image.
I used the mp.Pool(NUM_FUNCTIONS), but each process uses the …
-
## Describe the bug
I am currently building the model from the source for the model - `meta-llama/Meta-Llama-3-8B-Instruct`:
```
ckpt_path = checkpoints[get_model_parallel_rank()]
checkpoint = tor…
-
This was discussed fairly thoroughly in #426, the issue to add GPU support, but that's been fixed, so figured I'd open a new issue to keep this particular feature visible.
I have multiple GPUs, and…