Mostly a bit of code cleanup around sparsities. KL Sparsity should really only be applied to sigmoid outputs... I could put in some kind of warning but I might deprecate it anyway because it seems rarely used.
Also added back in a shallow vs deep autoencoder, where basically the decoder is pre or post the first couple FC layers. Still not having much luck with the deep autoencoder, but now that simulation performance is way better, I should make a bigger effort.
Mostly a bit of code cleanup around sparsities. KL Sparsity should really only be applied to sigmoid outputs... I could put in some kind of warning but I might deprecate it anyway because it seems rarely used.
Also added back in a shallow vs deep autoencoder, where basically the decoder is pre or post the first couple FC layers. Still not having much luck with the deep autoencoder, but now that simulation performance is way better, I should make a bigger effort.