speech evaluate terminate by 97% without result

deepgram / kur

Descriptive Deep Learning

Apache License 2.0

816 stars 107 forks source link

speech evaluate terminate by 97% without result #18

Closed liaoweiguo closed 7 years ago

liaoweiguo commented 7 years ago

DL 50p(4.5GB) trainning data, and change batch_size to 8, while training no validating result, after 6 epochs, "kur -v evaluate speech.yml" give me next result:

... [DEBUG 2017-02-24 10:56:07,746 kur.providers.batch_provider:185] Next batch of data has been prepared. Evaluating: 97%|█████████████████████████████▏| 264/271 [00:30<00:00, 10.79samples/s] sama@sama:~/ex/kur/speech$

No result!

any hints will be helpful

liaoweiguo commented 7 years ago

Importing new trainning data need a clean log/weights envioment? Retrainning for zero?

ajsyp commented 7 years ago

Not sure if you intended to close this. If so, what was your fix? If not, please post your Kurfile and we can take a look.

liaoweiguo commented 7 years ago

Download more data and using previous training result(log weights ..), and fail to validate. I have to purge data and retrainning from zero. It works but I donot know why. It's too time-consuming so I just close it