3D Autoencoder learnt only 1 dimension well

[cherepanovic]

given: an AE with 3D input and 3D output

unfortunately, the model learned only the first dimension well. I don't understand what the reason is, need your advice!

orange is the reconstruction, blue is the original data orange is reconstruction, blue is the original data

here is the model:

print(data.shape)
(1140, 3, 200)

input_dimension = 200

input_sample = keras.Input(shape=(3,input_dimension))

hidden1 = layers.Dense(150, activation='relu')(input_sample)

hidden2 = layers.Dense(100, activation='relu')(hidden1)
hidden3 = layers.Dense(50, activation='relu')(hidden2)
hidden4 = layers.Dense(100, activation='relu')(hidden3)
hidden5 = layers.Dense(150, activation='relu')(hidden4)

decoded = layers.Dense(200, activation='sigmoid')(hidden5)

autoencoder = keras.Model(input_sample, decoded)

autoencoder.compile(optimizer='adam', loss='binary_crossentropy')
autoencoder.summary()

Model: "functional_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (InputLayer)         [(None, 3, 200)]          0         
_________________________________________________________________
dense (Dense)                (None, 3, 150)            30150     
_________________________________________________________________
dense_1 (Dense)              (None, 3, 100)            15100     
_________________________________________________________________
dense_2 (Dense)              (None, 3, 50)             5050      
_________________________________________________________________
dense_3 (Dense)              (None, 3, 100)            5100      
_________________________________________________________________
dense_4 (Dense)              (None, 3, 150)            15150     
_________________________________________________________________
dense_5 (Dense)              (None, 3, 200)            30200     
=================================================================
Total params: 100,750
Trainable params: 100,750
Non-trainable params: 0

autoencoder.fit(data, data,
                epochs=500,
                batch_size=256,
                shuffle=True,)

[sanatmpa1]

@cherepanovic,

Can you share standalone code or colab gist to reproduce the issue? Thanks!

[cherepanovic]

@sanatmpa1 Hello Sanat,

here is the link to jupyter notebook https://github.com/cherepanovic/omwtuss/blob/master/3d_ae.ipynb .

the dataset is very small at the moment which means that the NN even should be able to overfit the data. Unfortunately, I am not allowed to share the data

[cherepanovic]

I mixed up the input, I took data instead of data_