search

ajbrock / BigGAN-PyTorch

The author's officially unofficial PyTorch BigGAN implementation.
MIT License
2.86k stars 476 forks source link

How to test on only one GPU? #21

Closed VoiceBeer closed 5 years ago

VoiceBeer commented 5 years ago

Hi, we are a group of students and is reproducing this BigGAN model for our coursework, one question is that we only have one GPU on the Colab, and is wondering how to modify the model, BTW, we are trying to use another dataset, there are some problems too. Hope to get your reply, really appreciate it :).

VoiceBeer commented 5 years ago

Sorry I didn't find that there is "Using your own dataset" in the README file, but still wondering is it possible to use only one GPU.

christegho commented 5 years ago

You should be able to find example commands on how to train your own model in the folder scripts. On colab, with one GPU, you should be able to train a model on a reduced batch size, I think a batch size 24 for 128x128 image outputs should work. You might also need to reduce the num_workers. You can increase num_G_accumulations and num_D_accumulations to remedy for the reduced batch size.

VoiceBeer commented 5 years ago

Thank you very much @christegho ! We are now working on how to use our own dataset, will try to modify the batch size and the iteration in the late work, and also thank you for your advice on the accumulations.

danielhuoo commented 5 years ago

You should be able to find example commands on how to train your own model in the folder scripts. On colab, with one GPU, you should be able to train a model on a reduced batch size, I think a batch size 24 for 128x128 image outputs should work. You might also need to reduce the num_workers. You can increase num_G_accumulations and num_D_accumulations to remedy for the reduced batch size.

Thanks for your reply. If we use our own dataset, I mean, we have a large number of images, what's the structure of folders should have?

christegho commented 5 years ago

in the directory with all the scripts for training, create a folder data. In data, create another folder with the name of your dataset, for example imagenet.

The folder imagenet will have a folder for every class, containing all images for each class.

ajbrock commented 5 years ago

As mentioned above, the amount of compute/VRAM you have determines the max batch size you can use, with bigger being better (up to a proportion of the size/complexity of your dataset). You can always spoof bigger batches with gradient accumulation, assuming you can fit at least a small batch into your setup.

VoiceBeer commented 5 years ago

thank you guys, very helpful 😆 @christegho @ajbrock