-
### 🐛 Describe the bug
amp static shape cpp wrapper
suite
name
thread
batch_size_new
speed_up_new
inductor_new
eager_new
compilation_…
-
I will refer to the [issue](https://github.com/huggingface/accelerate/issues/2027) I opened on the accelerate github, because I am unsure if this is an accelerate, Deep Speed or torch issue. All of th…
-
## 🐛 Bug
torch_cluster fails in DynamicEdgeConv when parallelising over multiple GPUs.
Training script with network architecture:
https://github.com/lgray/deepjet-geometric/blob/master/exam…
-
## Description
Show a GPU attached to an OpenFaaS function in Kubernetes
## Background
We have several users using Python for data-science where GPU acceleration is available. From the invest…
-
### 🐛 Describe the bug
Hello, when I am using DDP to train a model, I found that using multi-task loss and gradient checkpointing at the same time can lead to gradient synchronization failure betwe…
-
-
-
Thank-you for this repo. It is a great addition to TensorRT.
I use Windows 11, Visual Studio 2022 and cmake for c++ development. I would like to be able to build the c++ folder using just these too…
-
When running on 2 or 3 GPU's everything works fine. When running on 4 GPU's however, during the first epoch:
```
Epoch step: 1 Loss 9.134843 Tokens per Sec: 161.252618
Traceback (most recent call l…
-
The example is running on the NCCL backend for distributed GPU settings. I'm wondering if it can profile correctly on a multi-node (multiple CPU servers) distributed CPU settings with Gloo backend?
…