Two-Layer Convolutional Autoencoder does not learn anything useful

[maxkohlbrenner]

When training a deeper convolutional autoencoder with max-pooling, the reconstructions as visualized by the visualize_ae_representation function are all zero / black, this might be a problem of the visualization itself because the error still changes during the training.

Example setup:

filter_dims = [(7,7), (3,3)] hidden_channels = [4,6] use_max_pooling = True activation_function = 'relu'

[maxkohlbrenner]

The CAE actually learns nothing useful, the problem is not the visualization but the learning itself.

[maxkohlbrenner]

Current Problem is: a CAE with only one convolutional layer and max-pooling can succesfully reconstruct the input image, the general structure seems to be correct. However a two-layer CAE does not seem to learn anything useful to reconstruct the original image.

Parameters used for the CAE with two conv layers: filter_dims = [(5,5), (5,5)] # [(5,5), (5,5)] hidden_channels = [5, 5] # [32,64] use_max_pooling = True strides = None # other strides should not work yet activation_function = 'rect' batch_size = 100

[maxkohlbrenner]

A two layer training works when initializing the bias terms with smaller values (before: 0.1, now: 0.001) and using a sigmoid activation function. Deeper training with relu functions is not working yet. Opening a new issue for this

[maxkohlbrenner]

Actually, the sigmoid function seems to make the difference. With a sigmoid activation throughout the whole network, the autoencoder works fine even with initial bias values of 0.1