the visualization of output of the sparse encoder is wrong

[Eugenie-uv]

When I run the code from sparse autoencoder to sparse autoencoder, the visualization of output of the sparse encoder is wrong. And do does the deep autoencoder. Refer to https://blog.keras.io/building-autoencoders-in-keras.html

Here is the simple autoencoder output: ![simple_autoencoder_output] (https://cloud.githubusercontent.com/assets/17468922/24539370/cf146466-1620-11e7-9660-0c0f4ca72ff9.png)

The sparse autoencoder output:

The deep autoencoder output: http://www.jianshu.com/p/5a64c3ff52d9

My sparse autoencoder code is here, I just add the avtivity_regularizer to a simple autoencoder. ` from keras.layers import Input, Dense from keras.models import Model import matplotlib.pyplot as plt import numpy as np from keras import regularizers

encoding_dim = 32 input_img = Input(shape=(784,))

encoded = Dense(encoding_dim, activation='relu')(input_img)

encoded = Dense(encoding_dim, activation='relu', activity_regularizer=regularizers.l1(10e-5))(input_img)
decoded = Dense(784, activation='sigmoid')(encoded)
autoencoder = Model(input=input_img, output=decoded)

encoder = Model(input=input_img, output=encoded) encoded_input = Input(shape=(encoding_dim,)) decoder_layer = autoencoder.layers[-1] decoder = Model(input=encoded_input, output=decoder_layer(encoded_input))

autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')

f = np.load('mnist.npz') x_train = f['x_train'] y_train = f['y_train'] x_test = f['x_test'] y_test = f['y_test'] f.close()

x_train = x_train.astype('float32')/255. x_test = x_test.astype('float32')/255. x_train = x_train.reshape((len(x_train), np.prod(x_train.shape[1:]))) x_test = x_test.reshape((len(x_test), np.prod(x_test.shape[1:])))

print x_train.shape print x_test.shape autoencoder.fit(x_train, x_train, nb_epoch=3, batch_size=256, shuffle=True, validation_data=(x_test,x_test))

encoded_imgs = encoder.predict(x_test) decoded_imgs = decoder.predict(encoded_imgs) n =10 plt.figure(figsize=(20,4)) for i in range(n): ax = plt.subplot(2, n, i+1) plt.imshow(x_test[i].reshape(28,28)) ax = plt.subplot(2, n, i+1+n) plt.imshow(decoded_imgs[i].reshape(28,28)) plt.show()`

[cnheider]

To reproduce you need additional epochs, the 0.10 loss is reported for 100 epochs. However the activity regularization seems to be way off in the blog entry. l1(10e-9) worked just fine for me with a train loss of 0.99 and test of 0.97.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 30 days if no further activity occurs, but feel free to re-open a closed issue if needed.

[DerekGrant]

l1(10e-7) is much better. If you use l1(10e-9) , the loss will decrease, but the sparsity is worse. The encoded_imgs.mean increase and your regularization is meanless @cnHeider