Output problem when changed the input and output sequence lengths

Nemzy / language-translation

Neural machine translator for English2German translation.

MIT License

96 stars 57 forks source link

Output problem when changed the input and output sequence lengths #7

Open vinaykumarbundla opened 6 years ago

vinaykumarbundla commented 6 years ago

Hi Nemzy, I trained the code with same input_seq_length=15 , output_seq_len = 17 ,when I tested for small input sentences (which contains 6 words ) its giving correct output .If I gave more than 8 words then output is not getting correctly . And one more i changed input_seq_length=21 output_seq_len=23 and tested then also same problem that is for smaller sentences its working properly and for bigger sentences its not working .can I know the reason for this problem?

Nemzy commented 6 years ago

Hi @vinaykumarbundla , Well this could happen for few reasons:

Dataset doesn't contain enough long sentences
RNN architecture is not deep/wide enough
RNNs generally have trouble to remember longer sequences (but I think this is not the case here), so bidirectional RNN could help

Also take a look at dynamic RNNs which are now available in new Tensorflow version.

vinaykumarbundla commented 6 years ago

Thanks for the Reply, I need one more clarification about the code and dataset,In my own dataset I have minimum=5 words,Average = 15 words and Maximum =40 words,Suppose If I made change the Input seq len from 15 to 30,Output seq len from 17 to 32 , Will I get better accuracy at the output when I test it with 15 words?

Nemzy commented 6 years ago

Hi @vinaykumarbundla ,

Sorry for this late reply. To improve accuracy of your model the best thing to do is bucketing, basically you should split sentences in buckets by their length (for example: Bucket1 (1, 10), Bucket2 (10, 15), Bucket3 (15, 20), Bucket4 (20, 30), Bucket5 (30, 40)) and with every bucket train different model.

Or check dynamic RNNs cause then you won't be restricted to use fixed length input sequences.