neccam / nslt

Neural Sign Language Translation (CVPR'18)
Apache License 2.0
149 stars 41 forks source link

Problem while running nmt.py during training #22

Closed PoojaSharma15 closed 4 years ago

PoojaSharma15 commented 5 years ago

Trainable variables

conv1/weights:0, (11, 11, 3, 96), /device:GPU:0 conv1/biases:0, (96,), /device:GPU:0 conv2/weights:0, (5, 5, 48, 256), /device:GPU:0 conv2/biases:0, (256,), /device:GPU:0 conv3/weights:0, (3, 3, 256, 384), /device:GPU:0 conv3/biases:0, (384,), /device:GPU:0 conv4/weights:0, (3, 3, 192, 384), /device:GPU:0 conv4/biases:0, (384,), /device:GPU:0 conv5/weights:0, (3, 3, 192, 256), /device:GPU:0 conv5/biases:0, (256,), /device:GPU:0 fc6/weights:0, (9216, 4096), /device:GPU:0 fc6/biases:0, (4096,), /device:GPU:0 fc7/weights:0, (4096, 4096), /device:GPU:0 fc7/biases:0, (4096,), /device:GPU:0 embeddings/decoder/embedding_decoder:0, (2891, 32), /device:GPU:0 dynamic_seq2seq/encoder/rnn/multi_rnn_cell/cell_0/basic_lstm_cell/kernel:0, (4128, 128), /device:GPU:0 dynamic_seq2seq/encoder/rnn/multi_rnn_cell/cell_0/basic_lstm_cell/bias:0, (128,), /device:GPU:0 dynamic_seq2seq/encoder/rnn/multi_rnn_cell/cell_1/basic_lstm_cell/kernel:0, (64, 128), /device:GPU:0 dynamic_seq2seq/encoder/rnn/multi_rnn_cell/cell_1/basic_lstm_cell/bias:0, (128,), /device:GPU:0 dynamic_seq2seq/decoder/memory_layer/kernel:0, (32, 32), /device:GPU:0 dynamic_seq2seq/decoder/attention/multi_rnn_cell/cell_0/basic_lstm_cell/kernel:0, (96, 128), /device:GPU:0 dynamic_seq2seq/decoder/attention/multi_rnn_cell/cell_0/basic_lstm_cell/bias:0, (128,), /device:GPU:0 dynamic_seq2seq/decoder/attention/multi_rnn_cell/cell_1/basic_lstm_cell/kernel:0, (64, 128), /device:GPU:0 dynamic_seq2seq/decoder/attention/multi_rnn_cell/cell_1/basic_lstm_cell/bias:0, (128,), /device:GPU:0 dynamic_seq2seq/decoder/attention/attention_layer/kernel:0, (64, 32), /device:GPU:0 dynamic_seq2seq/decoder/output_projection/kernel:0, (32, 2891), /device:GPU:0

creating eval graph ...

num_layers = 2, num_residual_layers=0 cell 0 LSTM, forget_bias=1 DeviceWrapper, device=/gpu:0 cell 1 LSTM, forget_bias=1 DeviceWrapper, device=/gpu:0 cell 0 LSTM, forget_bias=1 DeviceWrapper, device=/gpu:0 cell 1 LSTM, forget_bias=1 DeviceWrapper, device=/gpu:0 start_decay_step=0, learning_rate=1e-05, decay_steps 10000, decay_factor 0.98

Trainable variables

conv1/weights:0, (11, 11, 3, 96), /device:GPU:0 conv1/biases:0, (96,), /device:GPU:0 conv2/weights:0, (5, 5, 48, 256), /device:GPU:0 conv2/biases:0, (256,), /device:GPU:0 conv3/weights:0, (3, 3, 256, 384), /device:GPU:0 conv3/biases:0, (384,), /device:GPU:0 conv4/weights:0, (3, 3, 192, 384), /device:GPU:0 conv4/biases:0, (384,), /device:GPU:0 conv5/weights:0, (3, 3, 192, 256), /device:GPU:0 conv5/biases:0, (256,), /device:GPU:0 fc6/weights:0, (9216, 4096), /device:GPU:0 fc6/biases:0, (4096,), /device:GPU:0 fc7/weights:0, (4096, 4096), /device:GPU:0 fc7/biases:0, (4096,), /device:GPU:0 embeddings/decoder/embedding_decoder:0, (2891, 32), dynamic_seq2seq/encoder/rnn/multi_rnn_cell/cell_0/basic_lstm_cell/kernel:0, (4128, 128), /device:GPU:0 dynamic_seq2seq/encoder/rnn/multi_rnn_cell/cell_0/basic_lstm_cell/bias:0, (128,), /device:GPU:0 dynamic_seq2seq/encoder/rnn/multi_rnn_cell/cell_1/basic_lstm_cell/kernel:0, (64, 128), /device:GPU:0 dynamic_seq2seq/encoder/rnn/multi_rnn_cell/cell_1/basic_lstm_cell/bias:0, (128,), /device:GPU:0 dynamic_seq2seq/decoder/memory_layer/kernel:0, (32, 32), dynamic_seq2seq/decoder/attention/multi_rnn_cell/cell_0/basic_lstm_cell/kernel:0, (96, 128), /device:GPU:0 dynamic_seq2seq/decoder/attention/multi_rnn_cell/cell_0/basic_lstm_cell/bias:0, (128,), /device:GPU:0 dynamic_seq2seq/decoder/attention/multi_rnn_cell/cell_1/basic_lstm_cell/kernel:0, (64, 128), /device:GPU:0 dynamic_seq2seq/decoder/attention/multi_rnn_cell/cell_1/basic_lstm_cell/bias:0, (128,), /device:GPU:0 dynamic_seq2seq/decoder/attention/attention_layer/kernel:0, (64, 32), /device:GPU:0 dynamic_seq2seq/decoder/output_projection/kernel:0, (32, 2891), /device:GPU:0

creating infer graph ...

num_layers = 2, num_residual_layers=0 cell 0 LSTM, forget_bias=1 DeviceWrapper, device=/gpu:0 cell 1 LSTM, forget_bias=1 DeviceWrapper, device=/gpu:0 cell 0 LSTM, forget_bias=1 DeviceWrapper, device=/gpu:0 cell 1 LSTM, forget_bias=1 DeviceWrapper, device=/gpu:0 WARNING:tensorflow:From C:\Users\Dell1\Anaconda3\envs\tsf\lib\site-packages\tensorflow\contrib\seq2seq\python\ops\beam_search_decoder.py:985: to_int64 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.cast instead. W1110 22:01:40.660759 21908 deprecation.py:323] From C:\Users\Dell1\Anaconda3\envs\tsf\lib\site-packages\tensorflow\contrib\seq2seq\python\ops\beam_search_decoder.py:985: to_int64 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.cast instead. start_decay_step=0, learning_rate=1e-05, decay_steps 10000, decay_factor 0.98

Trainable variables

conv1/weights:0, (11, 11, 3, 96), /device:GPU:0 conv1/biases:0, (96,), /device:GPU:0 conv2/weights:0, (5, 5, 48, 256), /device:GPU:0 conv2/biases:0, (256,), /device:GPU:0 conv3/weights:0, (3, 3, 256, 384), /device:GPU:0 conv3/biases:0, (384,), /device:GPU:0 conv4/weights:0, (3, 3, 192, 384), /device:GPU:0 conv4/biases:0, (384,), /device:GPU:0 conv5/weights:0, (3, 3, 192, 256), /device:GPU:0 conv5/biases:0, (256,), /device:GPU:0 fc6/weights:0, (9216, 4096), /device:GPU:0 fc6/biases:0, (4096,), /device:GPU:0 fc7/weights:0, (4096, 4096), /device:GPU:0 fc7/biases:0, (4096,), /device:GPU:0 embeddings/decoder/embedding_decoder:0, (2891, 32), dynamic_seq2seq/encoder/rnn/multi_rnn_cell/cell_0/basic_lstm_cell/kernel:0, (4128, 128), /device:GPU:0 dynamic_seq2seq/encoder/rnn/multi_rnn_cell/cell_0/basic_lstm_cell/bias:0, (128,), /device:GPU:0 dynamic_seq2seq/encoder/rnn/multi_rnn_cell/cell_1/basic_lstm_cell/kernel:0, (64, 128), /device:GPU:0 dynamic_seq2seq/encoder/rnn/multi_rnn_cell/cell_1/basic_lstm_cell/bias:0, (128,), /device:GPU:0 dynamic_seq2seq/decoder/memory_layer/kernel:0, (32, 32), dynamic_seq2seq/decoder/attention/multi_rnn_cell/cell_0/basic_lstm_cell/kernel:0, (96, 128), /device:GPU:0 dynamic_seq2seq/decoder/attention/multi_rnn_cell/cell_0/basic_lstm_cell/bias:0, (128,), /device:GPU:0 dynamic_seq2seq/decoder/attention/multi_rnn_cell/cell_1/basic_lstm_cell/kernel:0, (64, 128), /device:GPU:0 dynamic_seq2seq/decoder/attention/multi_rnn_cell/cell_1/basic_lstm_cell/bias:0, (128,), /device:GPU:0 dynamic_seq2seq/decoder/attention/attention_layer/kernel:0, (64, 32), /device:GPU:0 dynamic_seq2seq/decoder/output_projection/kernel:0, (32, 2891), WARNING:tensorflow:From D:\BTP\new\nslt\train.py:242: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.

W1110 22:01:40.832565 21908 deprecation_wrapper.py:119] From D:\BTP\new\nslt\train.py:242: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.

log_file=D:/BTP/new/Data\log_1573403500

created train model with fresh parameters, time 1.66s created infer model with fresh parameters, time 0.33s

288

b'KUESTE BISSCHEN WOLKE ABER neg-REGEN neg-HABEN' Traceback (most recent call last):

File "", line 1, in runfile('D:/BTP/new/nslt/nmt.py', wdir='D:/BTP/new/nslt')

File "C:\Users\Dell1\Anaconda3\envs\tsf\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 827, in runfile execfile(filename, namespace)

File "C:\Users\Dell1\Anaconda3\envs\tsf\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 110, in execfile exec(compile(f.read(), filename, 'exec'), namespace)

File "D:/BTP/new/nslt/nmt.py", line 378, in tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)

File "C:\Users\Dell1\Anaconda3\envs\tsf\lib\site-packages\tensorflow\python\platform\app.py", line 40, in run _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)

File "C:\Users\Dell1\Anaconda3\envs\tsf\lib\site-packages\absl\app.py", line 299, in run _run_main(main, args)

File "C:\Users\Dell1\Anaconda3\envs\tsf\lib\site-packages\absl\app.py", line 250, in _run_main sys.exit(main(argv))

File "D:/BTP/new/nslt/nmt.py", line 368, in main run_main(FLAGS, default_hparams, train_fn, inference_fn)

File "D:/BTP/new/nslt/nmt.py", line 361, in run_main train_fn(hparams, target_session=target_session)

File "D:\BTP\new\nslt\train.py", line 265, in train run_full_eval(model_dir, infer_model, infer_sess, eval_model, eval_sess, hparams, summary_writer, sample_src_data, sample_tgt_data)

File "D:\BTP\new\nslt\train.py", line 188, in run_full_eval run_sample_decode(infer_model, infer_sess, model_dir, hparams, summary_writer, sample_src_data, sample_tgt_data)

File "D:\BTP\new\nslt\train.py", line 135, in run_sample_decode _sample_decode(loaded_infer_model, global_step, infer_sess, hparams, infer_model.iterator, src_data, tgt_data, infer_model.src_placeholder, summary_writer)

File "D:\BTP\new\nslt\train.py", line 418, in _sample_decode nmt_outputs, attention_summary = model.decode(sess)

File "D:\BTP\new\nslt\model.py", line 409, in decode _, infersummary, , sample_words = self.infer(sess)

File "D:\BTP\new\nslt\model.py", line 397, in infer return sess.run([self.infer_logits, self.infer_summary, self.sample_id, self.sample_words])

File "C:\Users\Dell1\Anaconda3\envs\tsf\lib\site-packages\tensorflow\python\client\session.py", line 950, in run run_metadata_ptr)

File "C:\Users\Dell1\Anaconda3\envs\tsf\lib\site-packages\tensorflow\python\client\session.py", line 1173, in _run feed_dict_tensor, options, run_metadata)

File "C:\Users\Dell1\Anaconda3\envs\tsf\lib\site-packages\tensorflow\python\client\session.py", line 1350, in _do_run run_metadata)

File "C:\Users\Dell1\Anaconda3\envs\tsf\lib\site-packages\tensorflow\python\client\session.py", line 1370, in _do_call raise type(e)(node_def, op, message)

UnknownError: FileNotFoundError: [WinError 3] The system cannot find the path specified: b'KUESTE BISSCHEN WOLKE ABER neg-REGEN neg-HABEN' Traceback (most recent call last):

File "C:\Users\Dell1\Anaconda3\envs\tsf\lib\site-packages\tensorflow\python\ops\script_ops.py", line 209, in call ret = func(*args)

File "D:\BTP\new\nslt\utils\iterator_utils.py", line 73, in get_number_of_frames return np.int32(len([f for f in listdir(src) if isfile(join(src, f))]))

FileNotFoundError: [WinError 3] The system cannot find the path specified: b'KUESTE BISSCHEN WOLKE ABER neg-REGEN neg-HABEN'

 [[{{node PyFunc}}]]
 [[IteratorGetNext]]

while printing the "src" we get "b'KUESTE BISSCHEN WOLKE ABER neg-REGEN neg-HABEN'" which I suppose is errenous data.

Can someone help to locate the error.