search

chenhsuanlin / spatial-transformer-GAN

ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositing :eyeglasses: (CVPR 2018)
MIT License
334 stars 72 forks source link

Questions regarding reproduction of the glasses dataset #19

Closed sbrugman closed 5 years ago

sbrugman commented 5 years ago

I would like to reproduce the glasses dataset experiment as reported in your publication. The paper reports the number of wraps to be N=5. The code has the setting of wraps to be N=1.

Is it correct that this warpN setting should be set to 5 to reproduce? In this case, CUDA runs out of memory on the NVIDIA GTX 1070 (8 GB) I am using. Related to this, both the paper and the supplementary material do not report any hardware specifications. Could you share the details of the setup that was used during your experiments?

chenhsuanlin commented 5 years ago

The default value for opt.warpN is arbitrarily set because the value changes for every added generator; please refer to the training script on this. When the 5th generator is added, opt.warpN would be 5.

I had been training on an NVIDIA Titan Xp (12GB); the default setting would probably cause an out-of-memory error with an 8GB graphics card. A couple of alternatives to get around this would be to

  1. decrease the batch size and increase the training iterations (haven't tried this myself yet though), or
  2. modify the code to free the memory of the previous i-1 generators when training the ith, since the weights in the i-1 generators would be frozen and does not need backprop (I wrote the code in the current form to allow some more flexibility in modifications).
ipod825 commented 5 years ago

@chenhsuanlin Hi, I tried to run the glasses/train.sh. But to my surprise, even when warpN=5, the whole model still only occupied 2137 MB of GPU memory. All I did was modifying the data path in glasses/data.py. Did I miss something or is this also the case when you ran the glasses experiment.

And also, I observed that when warpN increased, the amount of memory used did not increase correspondingly.

chenhsuanlin commented 5 years ago

@ipod825 -- Yes, this is because the checkpoint size is relatively small (~50MB), so even 5 copies of the generator may not affect the GPU memory size allocated by Tensorflow.

@sbrugman -- I hadn't been observing the memory growth much because I never really had issues, but as pointed out, ST-GAN should be able to fit in an 8GB graphics card if you're running our provided training script. Please feel free to reopen if you still have further issues.