Full voice slt_arctic s1 run time estimate

[tomleo]

The run_demo.sh in merlin/egs/slt_arctic/s1 didn't take too long, however the run_full_voice.sh in the same folder has been running for 2+ hours.

2018-10-02 11:36:08,971 INFO     main.train_DNN: fine-tuning the DNN model
2018-10-02 11:46:07,568 INFO     main.train_DNN: epoch 1, validation error 164.205109, train error 171.843094  time spent 598.60
2018-10-02 11:56:20,268 INFO     main.train_DNN: epoch 2, validation error 162.035263, train error 167.465256  time spent 609.37
2018-10-02 12:05:49,562 INFO     main.train_DNN: epoch 3, validation error 160.646759, train error 165.351212  time spent 565.83
2018-10-02 12:15:10,348 INFO     main.train_DNN: epoch 4, validation error 159.667664, train error 163.792099  time spent 557.33
2018-10-02 12:24:53,109 INFO     main.train_DNN: epoch 5, validation error 158.914246, train error 162.596085  time spent 579.29
2018-10-02 12:34:09,344 INFO     main.train_DNN: epoch 6, validation error 158.357498, train error 161.629639  time spent 552.73
2018-10-02 12:43:43,367 INFO     main.train_DNN: epoch 7, validation error 157.970078, train error 160.816528  time spent 570.54
2018-10-02 12:53:01,244 INFO     main.train_DNN: epoch 8, validation error 157.713150, train error 160.107224  time spent 554.43
2018-10-02 13:02:23,884 INFO     main.train_DNN: epoch 9, validation error 157.554703, train error 159.469101  time spent 559.24
2018-10-02 13:12:13,769 INFO     main.train_DNN: epoch 10, validation error 157.464111, train error 158.881302  time spent 586.48
2018-10-02 13:23:40,499 INFO     main.train_DNN: epoch 11, validation error 158.883545, train error 160.502243  time spent 683.25
2018-10-02 13:33:35,028 INFO     main.train_DNN: epoch 12, validation error 157.393433, train error 157.557480  time spent 594.53

2018-10-02 13:43:37,730 INFO     main.train_DNN: epoch 13, validation error 156.932922, train error 155.984451  time spent 599.17
2018-10-02 13:53:05,391 INFO     main.train_DNN: epoch 14, validation error 156.587601, train error 155.074097  time spent 564.10
2018-10-02 14:02:29,066 INFO     main.train_DNN: epoch 15, validation error 156.396912, train error 154.566925  time spent 560.16
2018-10-02 14:12:13,959 INFO     main.train_DNN: epoch 16, validation error 156.324127, train error 154.294159  time spent 581.37
2018-10-02 14:21:51,042 INFO     main.train_DNN: epoch 17, validation error 156.175934, train error 154.149612  time spent 573.55

It's not completely clear to me what the error metrics or epoch counts mean. Is their a max number of passes (epochs?) or some way to understand how long this process will take to complete?

[simonkingedinburgh]

On 2 Oct 2018, at 19:27, Tom wrote:

 has been running for 2+ hours.

read the config file - for example:

warmup_epoch : 10 training_epochs : 25

if you don’t know what epochs are, or what validation error is, you probably want to look for introductory course on neural networks

-- Prof. Simon King Director of the Centre for Speech Technology Research Professor of Speech Processing University of Edinburgh,UK www.cstr.ed.ac.uk

The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.