./seq2seq.sh config/LibriSpeech/AST.yaml --train -v --purge
an error is raised:
06/11 06:44:24 step 177000 epoch 101 learning rate 0.001 step-time 1.064 loss nan
06/11 06:44:24 starting evaluation
06/11 06:45:50 dev bleu=0.00 ter=100.00 wer=100.00 bleu1=0.00 loss=nan penalty=0.000 ratio=0.071
06/11 06:45:50 saving model to models/LibriSpeech/AST/checkpoints
06/11 06:45:50 finished saving model
Traceback (most recent call last):
File "/home/zd/anaconda3/envs/tensorflow/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/zd/anaconda3/envs/tensorflow/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/zd/speech_translation/seq2seq/translate/main.py", line 306, in
main()
File "/home/zd/speech_translation/seq2seq/translate/main.py", line 300, in main
model.train(config)
File "/home/zd/speech_translation/seq2seq/translate/translation_model.py", line 472, in train
self.train_step(loss_function=loss_function, use_baseline=use_baseline, kwargs)
File "/home/zd/speech_translation/seq2seq/translate/translation_model.py", line 527, in train_step
update_baseline=True)
File "/home/zd/speech_translation/seq2seq/translate/seq2seq_model.py", line 231, in step
encoder_inputs, targets, input_length = self.get_batch(data)
File "/home/zd/speech_translation/seq2seq/translate/seq2seq_model.py", line 330, in getbatch
for encoder, inputs in zip(self.encoders, inputs)]
File "/home/zd/speech_translation/seq2seq/translate/seq2seqmodel.py", line 330, in
for encoder, inputs in zip(self.encoders, inputs)]
ValueError: setting an array element with a sequence.
Error in `python3': free(): invalid pointer: 0x000055f69f8a4b00
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7fb347e0d7e5]
/lib/x86_64-linux-gnu/libc.so.6(+0x8037a)[0x7fb347e1637a]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7fb347e1a53c]
/home/zd/anaconda3/envs/tensorflow/lib/python3.6/site-packages/numpy/core/_multiarray_umath.cpython-36m-x86_64-linux-gnu.so(+0x2e757)[0x7fb34604c757]
python3(+0xf1532)[0x55f5f1c2a532]
python3(+0xf1532)[0x55f5f1c2a532]
python3(+0xf1532)[0x55f5f1c2a532]
python3(+0xf12b7)[0x55f5f1c2a2b7]
python3(+0xf1147)[0x55f5f1c2a147]
python3(+0x198aa2)[0x55f5f1cd1aa2]
python3(PyObject_CallFinalizer+0xf9)[0x55f5f1cd1f19]
python3(PyObject_CallFinalizerFromDealloc+0x1b)[0x55f5f1cd1f9b]
python3(+0x19902e)[0x55f5f1cd202e]
python3(+0xf18a8)[0x55f5f1c2a8a8]
python3(+0x1993b1)[0x55f5f1cd23b1]
python3(+0xf12b7)[0x55f5f1c2a2b7]
python3(+0xf1147)[0x55f5f1c2a147]
python3(+0xf115d)[0x55f5f1c2a15d]
python3(+0xf115d)[0x55f5f1c2a15d]
python3(+0xf115d)[0x55f5f1c2a15d]
python3(PyDict_SetItem+0x3da)[0x55f5f1c6fe7a]
python3(PyDict_SetItemString+0x4f)[0x55f5f1c7878f]
python3(PyImport_Cleanup+0x99)[0x55
./seq2seq.sh config/LibriSpeech/AST.yaml --train -v --purge an error is raised: 06/11 06:44:24 step 177000 epoch 101 learning rate 0.001 step-time 1.064 loss nan 06/11 06:44:24 starting evaluation 06/11 06:45:50 dev bleu=0.00 ter=100.00 wer=100.00 bleu1=0.00 loss=nan penalty=0.000 ratio=0.071 06/11 06:45:50 saving model to models/LibriSpeech/AST/checkpoints 06/11 06:45:50 finished saving model Traceback (most recent call last): File "/home/zd/anaconda3/envs/tensorflow/lib/python3.6/runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "/home/zd/anaconda3/envs/tensorflow/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/home/zd/speech_translation/seq2seq/translate/main.py", line 306, in
main()
File "/home/zd/speech_translation/seq2seq/translate/main.py", line 300, in main
model.train(config)
File "/home/zd/speech_translation/seq2seq/translate/translation_model.py", line 472, in train
self.train_step(loss_function=loss_function, use_baseline=use_baseline, kwargs)
File "/home/zd/speech_translation/seq2seq/translate/translation_model.py", line 527, in train_step
update_baseline=True)
File "/home/zd/speech_translation/seq2seq/translate/seq2seq_model.py", line 231, in step
encoder_inputs, targets, input_length = self.get_batch(data)
File "/home/zd/speech_translation/seq2seq/translate/seq2seq_model.py", line 330, in getbatch
for encoder, inputs in zip(self.encoders, inputs)]
File "/home/zd/speech_translation/seq2seq/translate/seq2seqmodel.py", line 330, in
for encoder, inputs in zip(self.encoders, inputs)]
ValueError: setting an array element with a sequence.
Error in `python3': free(): invalid pointer: 0x000055f69f8a4b00
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7fb347e0d7e5]
/lib/x86_64-linux-gnu/libc.so.6(+0x8037a)[0x7fb347e1637a]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7fb347e1a53c]
/home/zd/anaconda3/envs/tensorflow/lib/python3.6/site-packages/numpy/core/_multiarray_umath.cpython-36m-x86_64-linux-gnu.so(+0x2e757)[0x7fb34604c757]
python3(+0xf1532)[0x55f5f1c2a532]
python3(+0xf1532)[0x55f5f1c2a532]
python3(+0xf1532)[0x55f5f1c2a532]
python3(+0xf12b7)[0x55f5f1c2a2b7]
python3(+0xf1147)[0x55f5f1c2a147]
python3(+0x198aa2)[0x55f5f1cd1aa2]
python3(PyObject_CallFinalizer+0xf9)[0x55f5f1cd1f19]
python3(PyObject_CallFinalizerFromDealloc+0x1b)[0x55f5f1cd1f9b]
python3(+0x19902e)[0x55f5f1cd202e]
python3(+0xf18a8)[0x55f5f1c2a8a8]
python3(+0x1993b1)[0x55f5f1cd23b1]
python3(+0xf12b7)[0x55f5f1c2a2b7]
python3(+0xf1147)[0x55f5f1c2a147]
python3(+0xf115d)[0x55f5f1c2a15d]
python3(+0xf115d)[0x55f5f1c2a15d]
python3(+0xf115d)[0x55f5f1c2a15d]
python3(PyDict_SetItem+0x3da)[0x55f5f1c6fe7a]
python3(PyDict_SetItemString+0x4f)[0x55f5f1c7878f]
python3(PyImport_Cleanup+0x99)[0x55