search

vineetjohn / linguistic-style-transfer

Neural network parametrized objective to disentangle and transfer style and content in text
Apache License 2.0
138 stars 33 forks source link

Decoder not terminating sentences #12

Closed vineetjohn closed 6 years ago

vineetjohn commented 6 years ago

No EOS (end of sentence) tokens are being predicted by the decoder.

vineetjohn commented 6 years ago

Trying tf.contrib.seq2seq.GreedyEmbeddingHelper

UPDATE: Doesn't help

vineetjohn commented 6 years ago

Trying Tensorflow NMT's strategy of using EOS instead of PAD to pad shorter sequences.

UPDATE: Doesn't help

vineetjohn commented 6 years ago

It seems like testing with larger decoder lengths is a good strategy. However the sequence loss method doesn't work for early training output that finishes before the input sequence. Fix to try: Need to clip the ground truth to the max time steps in a particular minibatch.

UPDATE: Doesn't work for inference

vineetjohn commented 6 years ago

Incrementing sequence lengths by 1 seems to train on eos tokens as well, for GreedyEmbeddingHelper, and helps predict variable length sentences. Works for beam search too.