The problem of multi GPU training

muzairkhattak / multimodal-prompt-learning

[CVPR 2023] Official repository of paper titled "MaPLe: Multi-modal Prompt Learning".

https://muzairkhattak.github.io/multimodal-prompt-learning/

MIT License

578 stars 43 forks source link

The problem of multi GPU training #68

Closed yjn8 closed 5 days ago

yjn8 commented 1 month ago

Hello!

I have a question to consult. You mentioned using a single NVIDIA A100 GPU for training, but the code seems to support multi GPU training. There may be issues when using multi GPU training, and the main issue may be using nn.DataParallel() for multi GPU parallel training, when using nn.ParameterList() to construct a parameter list simultaneously, there will be an empty error in the forward phase . The specific error is shown in the following figure. Looking forward to your reply and answers. 0屏幕截图 1屏幕截图

muzairkhattak commented 1 month ago

Hi @yjn8,

Thank you for showing interest in MaPLe!

Regarding your question,

We note that our repository currently do not support the training of MaPLe on multiple GPUs and one might face unexpected errors while using the repo for multiple GPUs. We will strongly recommend you to proceed with the single GPU training while using our repository.

Thank you and kind regards!

Sincerely, Muhammad Uzair

muzairkhattak commented 5 days ago

Hi @yjn8,

I am closing the issue now. Feel free to open it if there are still any questions!