cannot run training on FFHQ with batchsize 64 and image size 48

raywzy / ICT

High-Fidelity Pluralistic Image Completion with Transformers (ICCV 2021)

312 stars 43 forks source link

cannot run training on FFHQ with batchsize 64 and image size 48 #13

Open LigZhong opened 3 years ago

LigZhong commented 3 years ago

I have 8 V100 gpus and run the training code without error with image size 32 and batch size 64 but cuda out of memory occurs when I set image size to 48. Anyone met this?

raywzy commented 3 years ago

Hi, Are your V100 GPUs 32G?

LigZhong commented 3 years ago

Hi, Are your V100 GPUs 32G?

Yes, I used V100 GPUs 32G. I checked that no other process occupied the GPUs.

raywzy commented 3 years ago

Have you tried other batch size setting, like 48?