search

Navidfoumani / ConvTran

This is a PyTorch implementation of ConvTran
MIT License
123 stars 8 forks source link

About "Cuda out of Memory" for "EigenWorms" dataset #5

Closed firrice closed 5 months ago

firrice commented 5 months ago

Hi, the ConvTran method is great, and i have been doing experiments based on it. During my experiments, i have encounter a problem about "CUDA out of memory". Especially for "EigenWorms" dataset which has a large length of 17984, when excuting your source code to line75 in attention.py as depicted blow: image the dimension scale of q and k both are (16, 8, 17984, 2), and attn is (16, 8, 17984, 17984) respectively. In term of GPU memory, attn variable(float32) will take 16×8×17984×17984×32/4/1024/1024/1024=154.22G, which is surly a large space complexity. While in the paper, there was "All of our experiments were conducted using the PyTorch framework in Python on a computing system consisting of a single Nvidia A5000 GPU with 24GB of memory and an Intel(R) Core(TM) i9-10900K CPU.", so i wonder how to solve the problem above based on 24G GPU memory. Look forward to your reply!

Navidfoumani commented 5 months ago

Hello,

Thank you for your comment. Indeed, EigenWorms presents unique challenges due to its dimensions. There are a couple of options to consider for running it, and you can expect almost the same results with each approach: 1- Run it on the CPU. 2- Add stride and valid padding in the embedding layer to help reduce the input size. 3- Reduce the batch size.