search

farizrahman4u / seq2seq

Sequence to Sequence Learning with Keras
GNU General Public License v2.0
3.17k stars 845 forks source link

Large output layer #202

Open fortuin opened 7 years ago

fortuin commented 7 years ago

I am trying to use a SimpleSeq2Seq model to perform a simple translation task. The size of the target vocabulary is 15000 and therefore also the size of the output layer. When initializing the model, I get the error message:

ValueError: Cannot create a tensor proto whose content is larger than 2GB.

Is there a workaround for that?

0b01 commented 7 years ago

Yes. Use a smaller vocab size.

fortuin commented 7 years ago

This is not an extremely large vocabulary. I've trained seq2seq models with vocabulary sizes of 30,000 and more in TensorFlow without getting these errors. My question is therefore why this seq2seq model needs to initialize such a large tensor proto and which part of the code I would have to change to make it work like in the tf.contrib.seq2seq models.