search

rizkiarm / LipNet

Keras implementation of 'LipNet: End-to-End Sentence-level Lipreading'
MIT License
640 stars 228 forks source link

ValueError: output of generator should be a tuple `(x, y, sample_weight)` or `(x, y)`. Found: None #77

Open gzz666 opened 5 years ago

gzz666 commented 5 years ago

Loading dataset list from cache... Found 1582 videos for training. Found 395 videos for validation.

WARNING:tensorflow:From /home/gaozhenzhen/.local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version. Instructions for updating: Colocations handled automatically by placer. WARNING:tensorflow:From /home/gaozhenzhen/.local/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py:2681: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version. Instructions for updating: Please use rate instead of keep_prob. Rate should be set to rate = 1 - keep_prob. WARNING:tensorflow:From /home/gaozhenzhen/.local/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py:1062: calling reduce_prod_v1 (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version. Instructions for updating: keep_dims is deprecated, use keepdims instead WARNING:tensorflow:From /home/gaozhenzhen/.local/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py:1044: calling reduce_sum_v1 (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version. Instructions for updating: keep_dims is deprecated, use keepdims instead WARNING:tensorflow:From /home/gaozhenzhen/.local/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py:1008: calling reduce_max_v1 (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version. Instructions for updating: keep_dims is deprecated, use keepdims instead WARNING:tensorflow:From /home/gaozhenzhen/.local/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py:3256: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.cast instead. WARNING:tensorflow:From /home/gaozhenzhen/.local/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py:3236: to_int64 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.cast instead.

Layer (type) Output Shape Param #

the_input (InputLayer) (None, 75, 100, 50, 3) 0

zero1 (ZeroPadding3D) (None, 77, 104, 54, 3) 0

conv1 (Conv3D) (None, 75, 50, 25, 32) 7232

batc1 (BatchNormalization) (None, 75, 50, 25, 32) 128

actv1 (Activation) (None, 75, 50, 25, 32) 0

spatial_dropout3d_1 (Spatial (None, 75, 50, 25, 32) 0

max1 (MaxPooling3D) (None, 75, 25, 12, 32) 0

zero2 (ZeroPadding3D) (None, 77, 29, 16, 32) 0

conv2 (Conv3D) (None, 75, 25, 12, 64) 153664

batc2 (BatchNormalization) (None, 75, 25, 12, 64) 256

actv2 (Activation) (None, 75, 25, 12, 64) 0

spatial_dropout3d_2 (Spatial (None, 75, 25, 12, 64) 0

max2 (MaxPooling3D) (None, 75, 12, 6, 64) 0

zero3 (ZeroPadding3D) (None, 77, 14, 8, 64) 0

conv3 (Conv3D) (None, 75, 12, 6, 96) 165984

batc3 (BatchNormalization) (None, 75, 12, 6, 96) 384

actv3 (Activation) (None, 75, 12, 6, 96) 0

spatial_dropout3d_3 (Spatial (None, 75, 12, 6, 96) 0

max3 (MaxPooling3D) (None, 75, 6, 3, 96) 0

time_distributed_1 (TimeDist (None, 75, 1728) 0

bidirectional_1 (Bidirection (None, 75, 512) 3048960

bidirectional_2 (Bidirection (None, 75, 512) 1181184

dense1 (Dense) (None, 75, 28) 14364

softmax (Activation) (None, 75, 28) 0

Total params: 4,572,156.0 Trainable params: 4,571,772.0 Non-trainable params: 384.0

WARNING:tensorflow:From /home/gaozhenzhen/.local/lib/python2.7/site-packages/tensorflow/python/ops/math_grad.py:102: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Deprecated in favor of operator or tf.math.divide. 2019-03-31 10:04:31.454159: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2019-03-31 10:04:31.474657: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2208000000 Hz 2019-03-31 10:04:31.475609: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x55ee636f8570 executing computations on platform Host. Devices: 2019-03-31 10:04:31.475625: I tensorflow/compiler/xla/service/service.cc:158] StreamExecutor device (0): , Epoch 0: Curriculum(train: True, sentence_length: -1, flip_probability: 0.5, jitter_probability: 0.05) Epoch 1/20 Process Process-1: Traceback (most recent call last): File "/usr/lib/python2.7/multiprocessing/process.py", line 267, in _bootstrap self.run() File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run self._target(*self._args, *self._kwargs) File "/home/gaozhenzhen/.local/lib/python2.7/site-packages/keras/engine/training.py", line 606, in data_generator_task generator_output = next(self._generator) File "/cdrom/LipNet-master/lipnet/helpers/threadsafe.py", line 16, in next Traceback (most recent call last): File "/cdrom/LipNet-master/training/random_split/train.py", line 77, in train(run_name, 0, 20, 3, 100, 50, 75, 32, 50) File "/cdrom/LipNet-master/training/random_split/train.py", line 73, in train pickle_safe=True) File "/home/gaozhenzhen/.local/lib/python2.7/site-packages/keras/legacy/interfaces.py", line 88, in wrapper return func(args, **kwargs) File "/home/gaozhenzhen/.local/lib/python2.7/site-packages/keras/engine/training.py", line 1851, in fit_generator str(generator_output)) ValueError: output of generator should be a tuple (x, y, sample_weight) or (x, y). Found: None

Process finished with exit code 1

gzz666 commented 5 years ago

how to resolve this problems? thanks

DerekChia commented 5 years ago

I got it to work by first converting the GRID video to frames using scripts/extract_mouth_batch.py and made the changes proposed by @acarlos in #37 . Feel free to email me (email in profile) if you have difficulty running or getting started.

gzz666 commented 5 years ago

@DerekChia Sorry,I just saw your reply.Thank you very much

gzz666 commented 5 years ago

@DerekChia Hey，I got following error ： File "D:\Anaconda3\lib\site-packages\skvideo\io\io.py", line 240, in vreader assert _HAS_FFMPEG, "Cannot find installation of ffmpeg." AssertionError: Cannot find installation of ffmpeg.

I have installed ffmpeg too, and I still can`t fix it. How did you resolve the problem?

DerekChia commented 5 years ago

@gzz666 I'm running Ubuntu, not Windows, so I cannot help you with the FFMPEG installation. Sorry.

gzz666 commented 5 years ago

I solved this question.I'm runing also Ubuntu16.04. but I have another issue.Error loading videos...in the training.py do you know how to solve?

DerekChia commented 5 years ago

@gzz666 Can you paste the entire error message here?

gzz666 commented 5 years ago

@DerekChia Enumerating dataset list from disk... Error loading video: /cdrom/LipNet-master/training/random_split/datasets/video/s1/lgwtzn.mpg Error loading video: /cdrom/LipNet-master/training/random_split/datasets/video/s1/bwwn9a.mpg Error loading video: /cdrom/LipNet-master/training/random_split/datasets/video/s1/srbu9a.mpg Error loading video: /cdrom/LipNet-master/training/random_split/datasets/video/s1/lbak3a.mpg Error loading video: /cdrom/LipNet-master/training/random_split/datasets/video/s1/braf9s.mpg Error loading video: /cdrom/LipNet-master/training/random_split/datasets/video/s1/pgbk7s.mpg Error loading video: /cdrom/LipNet-master/training/random_split/datasets/video/s1/pbao8n.mpg Error loading video: /cdrom/LipNet-master/training/random_split/datasets/video/s1/sgbp5s.mpg Error loading video: /cdrom/LipNet-master/training/random_split/datasets/video/s1/lgbs8p.mpg ................... raceback (most recent call last): File "/cdrom/LipNet-master/training/random_split/train.py", line 75, in train(run_name, 0, 20, 3, 100, 50, 75, 32, 50) File "/cdrom/LipNet-master/training/random_split/train.py", line 71, in train pickle_safe=True) File "/home/ggg/anaconda2/lib/python2.7/site-packages/keras/legacy/interfaces.py", line 88, in wrapper return func(*args, **kwargs) File "/home/ggg/anaconda2/lib/python2.7/site-packages/keras/engine/training.py", line 1783, in fit_generator raise ValueError('When using a generator for validation data, ' ValueError: When using a generator for validation data, you must specify a value for validation_steps.

Process finished with exit code 1

This is he entire error message .thanks

gzz666 commented 5 years ago

@DerekChia I have resolved this problem.

gzz666 commented 5 years ago

@DerekChia can you tell me how to solved this tissue？from lipnet.lipreading.generators import RandomSplitGenerator ImportError: No module named lipnet.lipreading.generators

shoaibkh commented 5 years ago

@DerekChia I have resolved this problem.

Sir can you tell me that how you can resolve it i shall be very thank full to you

jiagnhaiyang commented 4 years ago

如何解决这个问题？谢谢

@DerekChia我已经解决了这个问题。

大神您好，您这个问题是怎么解决的？

jeonsanghun commented 4 years ago

I solved it, and I am converting from python 2.7 to 3.6 code