search

open-mmlab / mmaction

An open-source toolbox for action understanding based on PyTorch
https://open-mmlab.github.io/
Apache License 2.0
1.86k stars 353 forks source link

Series of erros (unexpected key, missing keys, IndexError) while trying to test the pre-trained model (recognizer) #23

Closed sparsh-b closed 5 years ago

sparsh-b commented 5 years ago

The following is the command & the errors that followed:

NOTE: The missing keys all have ".num_batches_tracked" in common.

sparsh/mmaction-master$ python tools/test_recognizer.py configs/ucf101/tsn_rgb_bninception.py modelzoo/tsn_2d_rgb_bninception_seg3_f1s1_b32_g8-98160339.pth Downloading: "https://open-mmlab.s3.ap-northeast-2.amazonaws.com/pretrain/third_party/bn_inception_caffe-ed2e8665.pth" to /home/sparsh/.cache/torch/checkpoints/bn_inception_caffe-ed2e8665.pth 100.0% unexpected key in source state_dict: fc.weight, fc.bias

missing keys in source state_dict: inception_4b_pool_proj_bn.num_batches_tracked, inception_5a_3x3_reduce_bn.num_batches_tracked, inception_4c_pool_proj_bn.num_batches_tracked, inception_5a_3x3_bn.num_batches_tracked, inception_4e_double_3x3_1_bn.num_batches_tracked, inception_5b_pool_proj_bn.num_batches_tracked, inception_4d_3x3_bn.num_batches_tracked, inception_3b_3x3_bn.num_batches_tracked, inception_5b_double_3x3_2_bn.num_batches_tracked, inception_4d_double_3x3_1_bn.num_batches_tracked, inception_4b_3x3_bn.num_batches_tracked, inception_5a_double_3x3_2_bn.num_batches_tracked, inception_5b_double_3x3_reduce_bn.num_batches_tracked, inception_4b_double_3x3_1_bn.num_batches_tracked, inception_4b_double_3x3_reduce_bn.num_batches_tracked, inception_4c_3x3_reduce_bn.num_batches_tracked, inception_3a_double_3x3_2_bn.num_batches_tracked, inception_5a_double_3x3_reduce_bn.num_batches_tracked, inception_3b_3x3_reduce_bn.num_batches_tracked, inception_5b_3x3_bn.num_batches_tracked, inception_5b_double_3x3_1_bn.num_batches_tracked, inception_3a_3x3_bn.num_batches_tracked, inception_4a_3x3_reduce_bn.num_batches_tracked, inception_3c_double_3x3_2_bn.num_batches_tracked, conv2_3x3_reduce_bn.num_batches_tracked, conv1_7x7_s2_bn.num_batches_tracked, inception_4c_double_3x3_reduce_bn.num_batches_tracked, inception_4c_double_3x3_1_bn.num_batches_tracked, inception_4d_double_3x3_2_bn.num_batches_tracked, inception_3b_1x1_bn.num_batches_tracked, inception_3c_double_3x3_1_bn.num_batches_tracked, inception_4a_double_3x3_2_bn.num_batches_tracked, inception_3b_double_3x3_2_bn.num_batches_tracked, inception_5a_1x1_bn.num_batches_tracked, inception_4c_1x1_bn.num_batches_tracked, inception_4c_3x3_bn.num_batches_tracked, inception_5b_3x3_reduce_bn.num_batches_tracked, inception_5a_pool_proj_bn.num_batches_tracked, inception_3b_pool_proj_bn.num_batches_tracked, inception_3b_double_3x3_1_bn.num_batches_tracked, inception_4c_double_3x3_2_bn.num_batches_tracked, inception_4b_double_3x3_2_bn.num_batches_tracked, inception_3a_double_3x3_1_bn.num_batches_tracked, inception_3a_1x1_bn.num_batches_tracked, inception_4a_1x1_bn.num_batches_tracked, inception_3a_pool_proj_bn.num_batches_tracked, inception_4a_double_3x3_1_bn.num_batches_tracked, inception_4b_1x1_bn.num_batches_tracked, inception_3c_3x3_bn.num_batches_tracked, inception_4d_1x1_bn.num_batches_tracked, inception_4e_3x3_bn.num_batches_tracked, inception_4d_double_3x3_reduce_bn.num_batches_tracked, inception_4d_3x3_reduce_bn.num_batches_tracked, inception_3c_double_3x3_reduce_bn.num_batches_tracked, inception_3a_3x3_reduce_bn.num_batches_tracked, inception_3b_double_3x3_reduce_bn.num_batches_tracked, inception_5a_double_3x3_1_bn.num_batches_tracked, inception_4e_double_3x3_2_bn.num_batches_tracked, inception_4a_3x3_bn.num_batches_tracked, inception_4e_double_3x3_reduce_bn.num_batches_tracked, conv2_3x3_bn.num_batches_tracked, inception_4e_3x3_reduce_bn.num_batches_tracked, inception_3c_3x3_reduce_bn.num_batches_tracked, inception_5b_1x1_bn.num_batches_tracked, inception_4a_pool_proj_bn.num_batches_tracked, inception_4b_3x3_reduce_bn.num_batches_tracked, inception_4d_pool_proj_bn.num_batches_tracked, inception_3a_double_3x3_reduce_bn.num_batches_tracked, inception_4a_double_3x3_reduce_bn.num_batches_tracked

[>>>>> ] 423/3783, 0.8 task/s, elapsed: 535s, ETA: 4246sTraceback (most recent call last):

File "tools/test_recognizer.py", line 122, in main() File "tools/test_recognizer.py", line 83, in main outputs = single_test(model, data_loader) File "tools/test_recognizer.py", line 20, in single_test for i, data in enumerate(data_loader): File "/home/sparsh/anaconda3/envs/venv/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 568, in next return self._process_next_batch(batch) File "/home/sparsh/anaconda3/envs/venv/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 608, in _process_next_batch raise batch.exc_type(batch.exc_msg) IndexError: Traceback (most recent call last): File "/home/sparsh/anaconda3/envs/venv/lib/python3.5/site-packages/torch/utils/data/_utils/worker.py", line 99, in _worker_loop samples = collate_fn([dataset[i] for i in batch_indices]) File "/home/sparsh/anaconda3/envs/venv/lib/python3.5/site-packages/torch/utils/data/_utils/worker.py", line 99, in samples = collate_fn([dataset[i] for i in batch_indices]) File "/media/Seagate_4TB2/sparsh/mmaction-master/mmaction/datasets/rawframes_dataset.py", line 271, in getitem gt_label=DC(to_tensor(record.label), stack=True, File "/media/Seagate_4TB2/sparsh/mmaction-master/mmaction/datasets/rawframes_dataset.py", line 25, in label return int(self._data[2]) IndexError: list index out of range

zhaoyue-zephyrus commented 5 years ago

Hi @sparsh-b

The missing keys with name ".num_batches_tracked" don't affect the outcome.

The error actually comes from the last line IndexError: list index out of range. Did you check the format of your filelist? It should have multiple lines, each of which have three elements split by space: $video_id $num_frame $label

sparsh-b commented 5 years ago

Thanks a lot for the reply @zhaoyue-zephyrus !

The rawframes filelist is in the said format, but the videos filelist is in the format of: $video_id $label

I'm attaching the screenshots of both. P.S: I'm using UCF101 Videos: image Rawframes: image

zhaoyue-zephyrus commented 5 years ago

@sparsh-b Please make sure that the Dataset type and the filelist type are consistent in your config. For example, if your dataset is RawFramesDataset, then the valid filelist should be those with name rawframes.

sparsh-b commented 5 years ago

The dataset_type in config is set to RawFramesDataset. But, i couldn't find filelist type in config file (Pls let me know where to check for filelist type.). The contents of the folder (data/ucf101) containing filelists looks like this: image (pls ignore the presence of the folders rawframes_24th & rawframes_25th_1)

zhaoyue-zephyrus commented 5 years ago

Please check this ann_file in https://github.com/open-mmlab/mmaction/blob/master/configs/ucf101/tsn_rgb_bninception.py#L38

sparsh-b commented 5 years ago

Thanks for your time & patience!

I've cheked. the value of ann_file in lines 38, 61, 82 match with those in my /configs/ucf101/tsn_rgb_bninception.py file.

zhaoyue-zephyrus commented 5 years ago

Since you are testing, then you only need to focus on L82 (for test). If you follow the tutorials in mmaction/data_tools/, then there should be no problem... Did you check that all lines in ucf101_val_split_1_rawframes.txt agree with the format?

sparsh-b commented 5 years ago

@zhaoyue-zephyrus I followed the mentioned tutorials.

Yeah, some lines of ucf101_val_split_1_rawframes.txt, ucf101_val_split_2_rawframes.txt, ucf101_val_split_3_rawframes.txt were not in the format(just came to know about it, had to write a script). The following are a sample of lines which were not there in the format: image This was from ucf101_val_split_1_rawframes.txt. Similar cases were also there in ucf101_val_split_2_rawframes.txt & ucf101_val_split_3_rawframes.txt( Btw, on what splits does testing happen on?)

I think the problem is: While I was extracting frames from the videos (using, data_tools/ucf101/extract_rgb_frames.sh ), the following message popped up a good number of times:

[Warning] length inconsistent! Early stop with X out of Y frames

and X was <= Y (Yes, for some videos, the warning popped up even when X was equal to Y). I suppose, for some videos, X was equal to 0 and this might have caused the issue. Please let me know if there is a way to solve my issue without needing to extract the frames again.

Note: extract_rgb_frames.sh was not there in data_tools/ucf101. So, I, wrote a data_tools/ucf101/extract_rgb_frames.sh similar to data_tools/kinetics400/extract_rgb_frames.sh. This looks as follows: image

Thanks a lot for helping me out!

zhaoyue-zephyrus commented 5 years ago

I did not come across this warning: I specifically test with the mentioned video v_PlayingSitar_g04_c04 with 427 corrected frames.

The case of X < Y is understandable due to the diversity of video en/de-coder. But it is weird to have X=0. If there is indeed no frame, I think maybe your video is broken. (Check it out here: https://github.com/open-mmlab/mmaction/blob/master/data_tools/build_rawframes.py#L26)

Also, I think it won't cost a lot to process there videos with empty output manually.

sparsh-b commented 5 years ago

I don't know what was the problem the first time(the video is not broken), but, tried extracting the frames from the same videos (only from those which didn't output frames the 1st time) & it worked fine, now. Getting a mean-accuracy of 86.25% (using only RGB frames).

zhaoyue-zephyrus commented 5 years ago

Good 👍