search

jalola / improved-wgan-pytorch

Improved WGAN in Pytorch
MIT License
439 stars 68 forks source link

Compare to the tensorflow version, this code cost too much time. #14

Closed qxm1995716 closed 4 years ago

qxm1995716 commented 4 years ago

Hi, I had test the tensorflow version and pytorch version of WGAN-GP and found that the pytorch version wasted more time. Is this normal or my test has something wrong? THANKS.

jalola commented 4 years ago

Hi, I checked the training between TF and Pytorch we did in the past. Here is the speed of both:

TF: ~3-4s / iter (I am not sure if I ran with 1 or 2 GPUs 1080Ti) on a private dataset Pytorch: 6.4s / iter (1 GPU 1080Ti) on LSUN dataset using Pytorch DataLoader

How about your testing?

One layer that we didn't optimise in this repo is DepthToSpace since Pytorch does not have this layer implemented yet. https://github.com/jalola/improved-wgan-pytorch/blob/master/models/wgan.py#L40

There is a new implementation here: https://discuss.pytorch.org/t/is-there-any-layer-like-tensorflows-space-to-depth-function/3487/15

How about you can try replacing it and let us know. Just be careful to the order of input and output channel

qxm1995716 commented 4 years ago

Hi Thanks for your reply, your advices are very helpful. As your told, I will check the DepthToSpace part and have more experiment. I think my code might has some problem that lead to the huge difference between pytorch version and tf version, and I will checking my code again. Best wishes!

------------------ 原始邮件 ------------------ 发件人: "Hung Nguyen"<notifications@github.com>; 发送时间: 2020年3月18日(星期三) 晚上10:44 收件人: "jalola/improved-wgan-pytorch"<improved-wgan-pytorch@noreply.github.com>; 抄送: "1004599472"<1004599472@qq.com>;"Author"<author@noreply.github.com>; 主题: Re: [jalola/improved-wgan-pytorch] Compare to the tensorflow version, this code cost too much time. (#14)

Hi, I checked the training between TF and Pytorch we did in the past. Here is the speed of both:

TF: ~3-4s / iter (I am not sure if I ran with 1 or 2 GPUs 1080Ti) on a private dataset Pytorch: 6.4s / iter (1 GPU 1080Ti) on LSUN dataset using Pytorch DataLoader

How about your testing?

One layer that we didn't optimise in this repo is DepthToSpace since Pytorch does not have this layer implemented yet. https://github.com/jalola/improved-wgan-pytorch/blob/master/models/wgan.py#L40

There is a new implementation here: https://discuss.pytorch.org/t/is-there-any-layer-like-tensorflows-space-to-depth-function/3487/15

How about you can try replacing it and let us know. Just be careful to the order of input and output channel

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.