search

d2l-ai / d2l-en

Interactive deep learning book with multi-framework code, math, and discussions. Adopted at 500 universities from 70 countries including Stanford, MIT, Harvard, and Cambridge.
https://D2L.ai
Other
23.92k stars 4.36k forks source link

Unify hyperparameters of all frameworks in DCGAN #1778

Open astonzhang opened 3 years ago

astonzhang commented 3 years ago

https://github.com/d2l-ai/d2l-en/blob/master/chapter_generative-adversarial-networks/dcgan.md

Currently the TF implementation (https://github.com/d2l-ai/d2l-en/pull/1760/files) uses a different set of hyperparameters:

@tab mxnet, pytorch

latent_dim, lr, num_epochs = 100, 0.005, 20 train(net_D, net_G, data_iter, num_epochs, lr, latent_dim)

@tab tensorflow

latent_dim, lr, num_epochs = 100, 0.0005, 40 train(net_D, net_G, data_iter, num_epochs, lr, latent_dim)

Increasing num_epochs to 40 doubles the execution time in TF. Let's unify hyperparameters across all the frameworks.

astonzhang commented 3 years ago

@biswajitsahoo1111 Could you take a look at it? Thanks.

terrytangyuan commented 3 years ago

Agreed. We should also double check whether the results across all frameworks are consistent when the same set of hyperparameters are used.

biswajitsahoo1111 commented 3 years ago

@astonzhang Let me try a few other sets of hyperparameters. Getting a common set of hyperparameters should be our goal. I also agree with @terrytangyuan on this.