Closed think-high closed 6 years ago
Looks like you failed at "SoftmaxWithLoss" layer. In your command, you have --num_labels=89, the layer name at the error is "gpu_1/last_out_L14", that mean you prediction layer has only 14 outputs at softmax, but you label goes out of scope e.g. label >=14. Then you hit this error. Check careful why you input --num_labels=89 why the net still create the last layer with 14 outputs.
The integer label of the video must start from 0, here happened some errors when I started from 1.
@dutran : Actually the --num_labels was set to 14 in the script I ran. I just copied the other script here by mistake. My bad. Will correct it now.
@murilovarges : I actually have my labels starting from 1. Let me try this.
Then, changing your labels to start from 0 as @murilovarges suggested will solve your issue.
Hey. So, I did that and the process is not getting aborted and the training is running. But these warnings are still constantly being generated:
E0612 23:00:22.106964 9409 video_decoder.cc:75] Insufficient data to determine video format
E0612 23:00:22.107067 9411 video_decoder.cc:75] Insufficient data to determine video format
E0612 23:00:22.106992 9412 video_decoder.cc:75] Insufficient data to determine video format
E0612 23:00:22.106918 9407 video_decoder.cc:75] Insufficient data to determine video format
E0612 23:00:22.107008 9413 video_decoder.cc:75] Insufficient data to determine video format
E0612 23:00:22.107035 9414 video_decoder.cc:75] Insufficient data to determine video format
Can you help me understand the reason and whether it is a problem or not?
Thanks
It looks like, your video does not have some meta data, decoder does not know how to decode.
This message is in https://github.com/pytorch/pytorch/blob/master/caffe2/video/video_decoder.cc#L75.
I guess can happen when the video is very small too.
Similar issue in https://github.com/facebookresearch/video-nonlocal-net/issues/12
Oh great. This is very helpful. Thanks a lot @dutran and @murilovarges . 👍
Hi, I am trying to fine tune the pre-trained(Kinetics) R2Plus1D model on my dataset. I created the train and test LMDB of my dataset like this:
The format of the csv files is:
org_video("the path of videos"),label("integer label of the video")
And then I run this to train the model:
But the process is getting aborted with these errors:
.......... .......... ......... ......... .........
Here is what stdout dump that I passed to a file:
Can you please help me out of this?
Thanks, Rahul Bhojwani