Model Tuning - Githubissues

collinarnett / protein_gan

Implementation of "Generative Modeling for Protein Structures" by Namrata Anand and Po-Ssu Huang

GNU General Public License v3.0

18 stars 6 forks source link

Model Tuning #7

Closed collinarnett closed 4 years ago

collinarnett commented 4 years ago

Currently there are a few hyper parameters that aren't mentioned in the paper that need to be optimized:

batch_size
num_epochs
random_seed Batch size heavily affects training and num_epochs can be tuned after focusing on batch size as a hyperparameter. Random Seed randomly affects training so it will remain static at 666 until the very end.

collinarnett commented 4 years ago

75c402dde337061f44189a1b32328763dcfafd59 added some new changes to better reflect paper for better training stability.

collinarnett commented 4 years ago

I've tried to implement changes to decrease loss of the generator but I can't seem to hack it without changing the entire architecture. One sided label smoothing definitely helps but is not perfect and introducing noice every 20 steps does not improve model learning.

collinarnett commented 4 years ago

Going to leave this now since the paper is now fully replicated.