baidu-research / ba-dls-deepspeech

Apache License 2.0
486 stars 174 forks source link

segmentation fault running the very first train model example ! #36

Open takingstock opened 6 years ago

takingstock commented 6 years ago

hi folks ..here's the system info .. keep getting a core dump ..no idea how to even investigate it as i tried grepping the directory for the info messages and couldn't find any files that had them :( OS

NAME="Amazon Linux AMI" VERSION="2016.09" ID="amzn" ID_LIKE="rhel fedora" VERSION_ID="2016.09"

python 2.7 CuDNN - 5.1.03 theano - 0.8.2 scipy - 0.19

command run : python train.py train_corpus.json validation_corpus.json model

output : Using Theano backend. Using gpu device 0: GRID K520 (CNMeM is disabled, cuDNN 5103) /usr/local/lib/python2.7/site-packages/Theano-0.8.2-py2.7.egg/theano/sandbox/cuda/init.py:600: UserWarning: Your cuDNN version is more recent than the one Theano officially supports. If you see any problems, try updating Theano or downgrading cuDNN to version 5. warnings.warn(warn) /usr/local/lib/python2.7/site-packages/Theano-0.8.2-py2.7.egg/theano/tensor/signal/downsample.py:6: UserWarning: downsample module has been moved to the theano.tensor.signal.pool module. "downsample module has been moved to the theano.tensor.signal.pool module.") 2018-02-21 10:16:24,445 INFO (data_generator) Reading description file: train_corpus.json for partition: train 2018-02-21 10:16:24,762 INFO (data_generator) Reading description file: validation_corpus.json for partition: validation 2018-02-21 10:16:26,368 INFO (model) Building gru model /home/ec2-user/speech/ba-dls-deepspeech/model.py:109: UserWarning: Update your Conv1D call to the Keras 2 API: Conv1D(1000, 11, kernel_initializer="glorot_uniform", name="conv1d", activation="relu", padding="valid", strides=2) activation='relu')(acoustic_input) /usr/local/lib64/python2.7/site-packages/keras/backend/theano_backend.py:1890: UserWarning: ['filter_dilation'] are now deprecated in tensor.nnet.abstract_conv.conv2d interface and will be ignored. filter_dilation=dilation_rate) /home/ec2-user/speech/ba-dls-deepspeech/model.py:119: UserWarning: Update your GRU call to the Keras 2 API: GRU(1000, return_sequences=True, kernel_initializer="glorot_uniform", activation="relu", name="rnn_1") return_sequences=True)(output) /home/ec2-user/speech/ba-dls-deepspeech/model.py:119: UserWarning: Update your GRU call to the Keras 2 API: GRU(1000, return_sequences=True, kernel_initializer="glorot_uniform", activation="relu", name="rnn_2") return_sequences=True)(output) /home/ec2-user/speech/ba-dls-deepspeech/model.py:119: UserWarning: Update your GRU call to the Keras 2 API: GRU(1000, return_sequences=True, kernel_initializer="glorot_uniform", activation="relu", name="rnn_3") return_sequences=True)(output) /home/ec2-user/speech/ba-dls-deepspeech/model.py:127: UserWarning: Update your Dense call to the Keras 2 API: Dense(29, kernel_initializer="glorot_uniform", activation="linear", name="dense") outputdim, name='dense', activation='linear', init=initialization, /home/ec2-user/speech/ba-dls-deepspeech/model.py:129: UserWarning: Update your Model call to the Keras 2 API: `Model(outputs=Reshape{3}..., inputs=/acoustic...)` model = Model(input=acoustic_input, output=network_output) 2018-02-21 10:16:48,449 INFO (model) Building train_fn

2018-02-21 10:18:39,389 INFO (model) Building val_fn 2018-02-21 10:18:48,256 INFO (data_generator) Iters: 336 Segmentation fault