Global Step exceeds 290000 but perplexity became stagnant. How much more time to finish the training ?

nicolas-ivanov / tf_seq2seq_chatbot

[unmaintained]

420 stars 203 forks source link

Global Step exceeds 290000 but perplexity became stagnant. How much more time to finish the training ? #22

Open dpanshu opened 7 years ago

dpanshu commented 7 years ago

I have run for almost 290000 global steps however the perplexity became fixated around 13 a long time ago. Should I kill the process or should I wait longer for process to exit itself. How much steps to wait in the second case? Thanks

7oranger commented 7 years ago

Have you solve the problem yet? I don't know when to finish training either.