How can I model an auto-encoder for text classification in keras?

[Imorton-zd]

def train_single_model(train,unlabel,test,train_label,test_label):

    # input shape: (nb_samples, 32)
    encoder = containers.Sequential([Dense(16, input_dim=32), Dense(8)])
    decoder = containers.Sequential([Dense(16, input_dim=8), Dense(32)])

    autoencoder = AutoEncoder(encoder=encoder, decoder=decoder, output_reconstruction=True)
    model = models.Sequential()
    model.add(autoencoder)

    # training the autoencoder:
    model.compile(optimizer='sgd', loss='mse')
    model.fit(train+unlabel, train+unlabel, nb_epoch=10)

    # predicting compressed representations of inputs:
    autoencoder.output_reconstruction = False  # the model has to be recompiled after modifying this property
    model.compile(optimizer='sgd', loss='mse')
    representations = model.predict(test)

    return representations

data = create_data('electronics','10_00')
rep = train_single_model(data[0],data[1],data[2],data[3],data[4])
print rep[0]

ValueError: dimension mismatch in args to gemm (128,120)x(32,16)->(128,16)
Apply node that caused the error: GpuDot22(GpuFromHost.0, <CudaNdarrayType(float32, matrix)>)
Toposort index: 11
Inputs types: [CudaNdarrayType(float32, matrix), CudaNdarrayType(float32, matrix)]
Inputs shapes: [(128, 120), (32, 16)]
Inputs strides: [(120, 1), (16, 1)]
Inputs values: ['not shown', 'not shown']
Outputs clients: [[GpuElemwise{Add}[(0, 0)](GpuDot22.0, GpuDimShuffle{x,0}.0)]]

The main part of my model is above. I have omitted the part of loading data. Would some one give me some suggestions?

[Imorton-zd]

@EderSantana @fchollet @tboquet @pasky

1. What is the shape and type of auto-encoder input? A sequence of word indexes like the dataset of imdb?
2. Is the architecture of this auto-encoder above right? And how can I model the denoising auto-encoder further?

[EderSantana]

ValueError: dimension mismatch in args to gemm means that your input maybe not have the correct size. Did you check that out?

Also, for text maybe you want to use a recurrent auto-encoder since input text phrases can have different lengths.

[Imorton-zd]

@EderSantana Can keras model the recurrent auto-encoder? In fact, I only want to implement the simple auto-encoder or denoising auto-encoder via keras. Would you give me an example? Thanks a lot.

[EderSantana]

We are working on a Variational Autoencoder example here: https://github.com/fchollet/keras/pull/1750 It should be ready soon.

[fengsky401]

I don't know [Dense(16, input_dim=32), Dense(8)] mean?

[Imorton-zd]

@fengsky401 It's the example in the keras documentation. In fact, I don't know exactly, either. @EderSantana Would you explain the meaning of [Dense(16, input_dim=32), Dense(8)]? Thanks a lot!

[EderSantana]

containers get a list of layers as input. You stack that list everytime you do model.add and model inherits from the Container class. So encoder = containers.Sequential([Dense(16, input_dim=32), Dense(8)]) initializes the list with all the layers you will need at once, instead of doing two calls to the add method. Note how there are two layers there Dense(16, input=32) and Dense(8).

[viksit]

@EderSantana could you shed some light on the uses of such a variational AE?

[EderSantana]

if the documentations here don't help: https://github.com/fchollet/keras/pull/1750/files i'd recommend reading this quick tutorial: http://vdumoulin.github.io/morphing_faces/ or watching this video: https://www.youtube.com/watch?v=P78QYjWh5sM