search

DLii-Research / deep-dna

A repository of deep learning models for DNA samples and sequences
1 stars 2 forks source link

Specifying the `training` flag in GANs causes mode collapse #4

Closed SirDavidLudwig closed 2 years ago

SirDavidLudwig commented 2 years ago

The implementation of the AC-GAN, perhaps even the vanilla GAN, suffers from mode collapse that does not occur with a manual custom training loop.

SirDavidLudwig commented 2 years ago

Removing the training flag from the train_step method resolves the mode collapse issue, but this is not an acceptable solution...

SirDavidLudwig commented 2 years ago

The issue is temporarily resolved with an improper solution.

SirDavidLudwig commented 2 years ago

https://github.com/tensorflow/addons/issues/2414

SirDavidLudwig commented 2 years ago

Custom spectral norm implementations still seem to result in mode collapse... Maybe it's integrated into the models incorrectly? Aside from the mode collapse, it trains OK...

EDIT: Or maybe not...

SirDavidLudwig commented 2 years ago

image 8 Epochs

SirDavidLudwig commented 2 years ago

Closing for now with the conclusion that spectral normalization is just bad for these types of training regimes. The original paper used the Geometric GAN training regime, which may have been more helpful.