search

ottokart / punctuator

An LSTM RNN for restoring missing punctuation in unsegmented text.
MIT License
79 stars 26 forks source link

Starting from a previous training #4

Open ngoel17 opened 9 years ago

ngoel17 commented 9 years ago

I think there is a need to allow iteration over previous training, and also to allow multiple threads during training. Overall the process is very slow. What are your thoughts about adapting http://www.rnnlm.org/ to this task. I think the LSTM cell is missing there.

ottokart commented 9 years ago

Thanks for the feedback! Do you mean the ability to continue training from the last saved state in case the program crashes?

Numpy with MKL or ATLAS/BLAS/LAPACK should be able to utilise multiple threads (I see 66% utilisation of my 4 cores with MKL). The rnnlm toolkit is single-threaded and does not support mini-batches, so I don't expect it to be much faster.

The example runs about 6000 wps in 1st phase on my laptop. How slow is it for you? Reducing the BPTT steps (from 5 to 3-4) should also speed things up and not hurt performance too much.