model overfitting

[ayyappa428]

Issue 1:

How to avoid overfitting? The model is getting overfitted.

Issue 2:

I trained the model with 5000 epochs with 100 MB dataset. I am not getting the exact output. For some sentences it is displaying correct output and for some sentences it is not displaying correct output.

For 5000 epochs

1.

Hi how are you (English)
comment comment (France)

2.

I went to office Je suis allés à bureau

3.

I am going to my home Je vais chez moi

I trained again with 8000 epochs i am not getting correct output.

1.

hai how are you Avez-vous comment quelle

2.

I went to office Je suis allé à bureau

3.

I am going to my home Je vais chez moi chez moi

Please suggest me for how many epochs i will get the correct output?

suggest any other alternatives to avoid overfitting

[Nemzy]

Hi @ayyappa428 ,

Well just a small correction, you didn't train the model with 5k epochs. 1 epoch is when your model sees a whole dataset once, so NUM_STEPS * BATCH_SIZE / NUM_SAMPLES = EPOCHS_TRAINED.

Steps I would try to increase accuracy of the model are:

• Train the model with more epochs, 10 at least (takes a lot of time!).
• Try different hyperparameters:
  • Deeper encoder and decoder networks (more layers).
  • Wider encoder and decoder networks (more neurons in layers).
  • Smaller learning rates.
  • etc.
• Reorder words in input sentences in reverse ("How are you -> you are How").

For overfitting just use standard techniques like regularization, dropout, bigger dataset etc.