search

taoxugit / AttnGAN

MIT License
1.33k stars 415 forks source link

How to run the network in parallel on multiple cards #89

Open activate-an opened 3 years ago

activate-an commented 3 years ago

When I try to increase the size of the output image, it will prompt that there is not enough memory, so how can I run the network on multiple cards in parallel to achieve a larger output?

fyw1999 commented 2 years ago

I have run the code in multiple GPUS successfully! You need to wrap models by DataParallel firstly. There are still some changes to be made in the code, beacuse when you use multiple cards, a batch will be separated and distributed on each GPU, after each part of a batch through models, they will be merged to a batch again, which will cause a lot of bugs in code. For example in RNN model, when different part of a batch through the model, the max value in argument cap_lens will be different.

AudityGhosh commented 8 months ago

@fyw1999 , could you please enlighten us with the changes specifically ?