search

rizkiarm / LipNet

Keras implementation of 'LipNet: End-to-End Sentence-level Lipreading'
MIT License
644 stars 230 forks source link

Video shape issues and Intel MKL FATAL ERROR: Cannot load libmkl_core.so. #19

Open deepakgupta1313 opened 7 years ago

deepakgupta1313 commented 7 years ago

I am running : ./train unseen_speakers

  1. When I don't do anything regarding vface and face_predictor path. I get the following error : `Using all available GPUs. Using TensorFlow backend. I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcublas.so.8.0 locally I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcudnn.so.5 locally I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcufft.so.8.0 locally I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcuda.so.1 locally I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcurand.so.8.0 locally

Enumerating dataset list from disk... Video /home/deepakgupta1313/Desktop/Deepak/Programs/Github/LipNet/training/unseen_speakers/datasets/train/s1/pbwc8n.mpg has incorrect shape (75, 360, 288, 3), must be (75, 100, 50, 3) Video /home/deepakgupta1313/Desktop/Deepak/Programs/Github/LipNet/training/unseen_speakers/datasets/train/s1/bwwn8p.mpg has incorrect shape (75, 360, 288, 3), must be (75, 100, 50, 3) Video /home/deepakgupta1313/Desktop/Deepak/Programs/Github/LipNet/training/unseen_speakers/datasets/train/s1/swio2p.mpg has incorrect shape (75, 360, 288, 3), must be (75, 100, 50, 3) Video /home/deepakgupta1313/Desktop/Deepak/Programs/Github/LipNet/training/unseen_speakers/datasets/train/s1/lgifzn.mpg has incorrect shape (75, 360, 288, 3), must be (75, 100, 50, 3) Video /home/deepakgupta1313/Desktop/Deepak/Programs/Github/LipNet/training/unseen_speakers/datasets/train/s1/srih2p.mpg has incorrect shape (75, 360, 288, 3), must be (75, 100, 50, 3) Video /home/deepakgupta1313/Desktop/Deepak/Programs/Github/LipNet/training/unseen_speakers/datasets/train/s1/brbm6n.mpg has incorrect shape (75, 360, 288, 3), must be (75, 100, 50, 3) Video /home/deepakgupta1313/Desktop/Deepak/Programs/Github/LipNet/training/unseen_speakers/datasets/train/s1/lbid3s.mpg has incorrect shape (75, 360, 288, 3), must be (75, 100, 50, 3) Video /home/deepakgupta1313/Desktop/Deepak/Programs/Github/LipNet/training/unseen_speakers/datasets/train/s1/bwim6p.mpg has incorrect shape (75, 360, 288, 3), must be (75, 100, 50, 3) Video /home/deepakgupta1313/Desktop/Deepak/Programs/Github/LipNet/training/unseen_speakers/datasets/train/s1/bbil5a.mpg has incorrect shape (75, 360, 288, 3), must be (75, 100, 50, 3) Video /home/deepakgupta1313/Desktop/Deepak/Programs/Github/LipNet/training/unseen_speakers/datasets/train/s1/swbp1a.mpg has incorrect shape (75, 360, 288, 3), must be (75, 100, 50, 3) Video /home/deepakgupta1313/Desktop/Deepak/Programs/Github/LipNet/training/unseen_speakers/datasets/train/s1/lbiq3a.mpg has incorrect shape (75, 360, 288, 3), must be (75, 100, 50, 3) Video /home/deepakgupta1313/Desktop/Deepak/Programs/Github/LipNet/training/unseen_speakers/datasets/train/s1/bbil4p.mpg has incorrect shape (75, 360, 288, 3), must be (75, 100, 50, 3) Video /home/deepakgupta1313/Desktop/Deepak/Programs/Github/LipNet/training/unseen_speakers/datasets/train/s1/prwq4p.mpg has incorrect shape (75, 360, 288, 3), must be (75, 100, 50, 3) `

  1. When I make the following changes in train.py. Please tell me if the change is correct? lip_gen = BasicGenerator(dataset_path=DATASET_DIR, vtype = "face", face_predictor_path="/home/deepakgupta1313/Desktop/Deepak/Programs/Github/LipNet/common/predictors/shape_predictor_68_face_landmarks.dat", minibatch_size=minibatch_size, img_c=img_c, img_w=img_w, img_h=img_h, frames_n=frames_n,

I get the following error: `Using all available GPUs. Using TensorFlow backend. I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcublas.so.8.0 locally I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcudnn.so.5 locally I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcufft.so.8.0 locally I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcuda.so.1 locally I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcurand.so.8.0 locally

Enumerating dataset list from disk... Intel MKL FATAL ERROR: Cannot load libmkl_core.so. ` Any help is appreciated.

rizkiarm commented 7 years ago

1) Remove LipNet/lipnet/lipreading/generators.py:95-100. It was a shape mismatch because the video wasn't in the form of 100x100px mouth-crop. You can either do that or just use the preprocessor code LipNet/scripts/extract_mouth_batch.py.

2) Your changes seemed to be correct. I think the error was caused by incorrectly installed dependencies or other libraries.

deepakgupta1313 commented 7 years ago

@rizkiarm Can you please elaborate how exactly to use LipNet/scripts/extract_mouth_batch.py to train. It is not clearly explained in the readme file.