search

Kaixhin / Autoencoders

Torch implementations of various types of autoencoders
MIT License
466 stars 77 forks source link

Seq2SeqAE converge output on test data #11

Open JohnBravo08am opened 4 years ago

JohnBravo08am commented 4 years ago

Tried to use Seq2SeqAE - it converges output on any test set.

Kaixhin commented 4 years ago

Sorry, I don't understand the issue?

JohnBravo08am commented 4 years ago

Hello, Kaixhin, sorry for short description. I've tried to use Seq2SeqAE with MNIST dataset using your script. The training process completed successfully with avg loss ~0.11, but when I try to evaluate trained model on any data from dataset the result is the same - all digits are reconstructed to the same output, which looked like mean of all input digits. I've tried other AE models and they worked fine, reconstructing input images at the end, but not Seq2SeqAE. Am I missing something, or is it expected behavior?

Kaixhin commented 4 years ago

No, as an autoencoder it should reconstruct the input. Unfortunately it has been a very long time since I've used Lua and Torch7, so I'm not going to attempt to fix this, but happy to take a pull request with a fix.

JohnBravo08am commented 4 years ago

Actually, I've faced the similar problem, using Torch7 and RNN package. I tried to train model with stack of 2 LSTM blocks (using Torch7 SeqLSTM implementation) to detect certain patterns in long data sequences. Trainig process runs fine, but during test the model gives the same output on any data sequence. Then, I found this example of autoencoder using RNN and SeqLSTM, but results are very similar - same output on any data.