-
I have a Titan X and have been running the Breakout simulation for over two days now and it's only `7%` through training and `nvidia-smi` is showing that it's only using `4-5%`. The README.md says tha…
-
### 기타
안녕하세요 궁금한게 있어서 여쭤봅니다.
코드를 돌리다가 -killed .py가 나오거나 1/1 epoch실행중에 broker down이 되어서 epoch를 넘어가기가 힘든데요 같은 코드를 다른 팀원이 돌릴때는 에러없이 돌아갔습니다 같은 --option을 사용해서 command입력했습니다 코드에도 gpu device 사용도 명시했구요
…
-
Hello! Thanks for this work! Here is my question!
I used my own dataset for training, with E-Branchformer as the encoder, Transformer as decoder.
Environment:
Ubuntu 20.02
pytorch.version=1.13.1+c…
-
I've been running inference with the provided pre-trained model, but I've noticed that it only runs on the CPU. I attempted to convert the code to run on a GPU; however, I get numerous runtime errors…
-
**Description**
A clear and concise description of what the bug is.
I'm running Triton Inference Server with vLLM backend as a container on Kubernetes.
I followed the [Triton metrics documentatio…
-
Hi,
Even though the net takes almost all of the GPU memory, the "Volatile GPU util" remains at 0. Therefore, I conclude that the net doesn't really run on GPU (the very slow training process also s…
-
### Anything you want to discuss about vllm.
Currently tp will not work on odd number of gpus, nor on the number that cannot divide 40 (e.g. 6 gpus). However, it is unavoidable to have these kinds of…
-
### Proposal to improve performance
Hi all, I'm running vicuna 13B on H100 using fp8, and I find when batch size is large, say 64 or 96, the gpu utilization is low, about 60%, this is an important ca…
-
First of all, thank you for creating this fantastic tool. I was wondering if there is a way to display the utilization of each GPU?
-
Hi
I wanted to have a model using NVIDIA H100 GPU instead of NVIDIA T4 GPU.
In the values possible I see:
```
llama-3.1-8b-instruct-fp8-l4:
enabled: false
features: ["TextGeneration"]
…