search

pclucas14 / pixel-cnn-pp

Pytorch Implementation of OpenAI's PixelCNN++
Other
345 stars 76 forks source link

Training time? #1

Closed mstribitaka closed 6 years ago

mstribitaka commented 6 years ago

Hey there,

Been trying out your code and just wondering what GPU card you're using and how long it takes? I'm using a GTX 1080 and it's taking around 1 hour for 1 epoch. Does that sound reasonable to you?

We had to reduce the batch size to 64 otherwise it ran out of memory.

Thanks!

pclucas14 commented 6 years ago

Hi,

thanks for your question. I'm running on a TitanX and it takes 50 minutes per epoch. There's currently a bug with DataParallel which prevents multi-gpu training, but once that's solved training time should go down!

Update : @mstribitaka I had a small memory overlap in the testing loop, you should now be able to double the batch size

pclucas14 commented 6 years ago

I'm gonna close this issue, as training time is reasonable on multi gpu. I tested on pytorch 0.2 and speed seems proportional to the amount of gpus available.