Finetuning HMDB51 failed

I have followed the tutorial on fine-tuning on HMDB51. However, I have encountered the following errors. I only modified the path to my data and split list. All other files were not changed.
Ignoring @/caffe2/caffe2/contrib/nccl:nccl_ops as it is not a valid file.
Ignoring @/caffe2/caffe2/contrib/gloo:gloo_ops as it is not a valid file.
Ignoring @/caffe2/caffe2/contrib/gloo:gloo_ops_gpu as it is not a valid file.
[E init_intrinsics_check.cc:43] CPU feature avx is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
[E init_intrinsics_check.cc:43] CPU feature avx2 is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
[E init_intrinsics_check.cc:43] CPU feature fma is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
INFO:train_net:Namespace(base_learning_rate=0.0002, batch_size=4, clip_length_of=8, clip_length_rgb=32, crop_size=112, cudnn_workspace_limit_mb=64, db_type='pickle', display_iter=10, do_flow_aggregation=0, epoch_size=40000, file_store_path='.', flow_data_type=0, frame_gap_of=2, gamma=0.1, get_video_id=0, gpus='1', input_type=0, is_checkpoint=0, model_depth=34, model_name='r2plus1d', num_channels=3, num_decode_threads=4, num_epochs=8, num_gpus=1, num_labels=51, pred_layer_name=None, pretrained_model='/home/jiashu/Documents/VMZ/model/r2.5d_d18_l16.pkl', profiling=0, sampling_rate_of=2, sampling_rate_rgb=1, scale_h=128, scale_w=171, step_epoch=2, test_data='/home/jiashu/Documents/VMZ/data/hmdb51_test01', train_data='/home/jiashu/Documents/VMZ/data/hmdb51_train01', use_cudnn=1, use_dropout=0, use_local_file=0, weight_decay=0.005)
INFO:model_builder:Validated: r2plus1d with 34 layers
INFO:model_builder:with input 32x112x112
INFO:train_net:Running on GPUs: [1]
INFO:train_net:Using epoch size: 40000
WARNING:root:[====DEPRECATE WARNING====]: you are creating an object from CNNModelHelper class which will be deprecated soon. Please use ModelHelper object with brew module. For more information, please refer to caffe2.ai and python/brew.py, python/brew_test.py for more information.
INFO:train_net:train set has 3570 examples
INFO:data_parallel_model:Parallelizing model for devices: [1]
INFO:data_parallel_model:Create input and model training operators
INFO:data_parallel_model:Model for GPU : 1
INFO:model_helper:outputing rgb data
INFO:model_builder:creating r2plus1d, depth=34...
INFO:video_model:Number of middle filters: 144
INFO:video_model:Number of middle filters: 144
INFO:video_model:Number of middle filters: 144
INFO:video_model:Number of middle filters: 144
INFO:video_model:Number of middle filters: 144
INFO:video_model:Number of middle filters: 144
INFO:video_model:Number of middle filters: 230
INFO:video_model:Number of middle filters: 288
INFO:video_model:Number of middle filters: 288
INFO:video_model:Number of middle filters: 288
INFO:video_model:Number of middle filters: 288
INFO:video_model:Number of middle filters: 288
INFO:video_model:Number of middle filters: 288
INFO:video_model:Number of middle filters: 288
INFO:video_model:Number of middle filters: 460
INFO:video_model:Number of middle filters: 576
INFO:video_model:Number of middle filters: 576
INFO:video_model:Number of middle filters: 576
INFO:video_model:Number of middle filters: 576
INFO:video_model:Number of middle filters: 576
INFO:video_model:Number of middle filters: 576
INFO:video_model:Number of middle filters: 576
INFO:video_model:Number of middle filters: 576
INFO:video_model:Number of middle filters: 576
INFO:video_model:Number of middle filters: 576
INFO:video_model:Number of middle filters: 576
INFO:video_model:Number of middle filters: 921
INFO:video_model:Number of middle filters: 1152
INFO:video_model:Number of middle filters: 1152
INFO:video_model:Number of middle filters: 1152
INFO:video_model:Number of middle filters: 1152
INFO:video_model:Number of middle filters: 1152
INFO:data_parallel_model:Adding gradient operators
INFO:data_parallel_model:Add gradient all-reduces for SyncSGD
INFO:data_parallel_model:Post-iteration operators for updating params
INFO:data_parallel_model:Add initial parameter sync
WARNING:data_parallel_model:------- DEPRECATED API, please use data_parallel_model.OptimizeGradientMemory() ----- 
WARNING:memonger:NOTE: Executing memonger to optimize gradient memory
INFO:memonger:Memonger memory optimization took 0.020285367965698242 secs
INFO:train_net:----- Create test net ----
WARNING:root:[====DEPRECATE WARNING====]: you are creating an object from CNNModelHelper class which will be deprecated soon. Please use ModelHelper object with brew module. For more information, please refer to caffe2.ai and python/brew.py, python/brew_test.py for more information.
INFO:train_net:test set has 1530 examples
INFO:data_parallel_model:Parallelizing model for devices: [1]
INFO:data_parallel_model:Create input and model training operators
WARNING:data_parallel_model:
WARNING:data_parallel_model:############# WARNING #############
WARNING:data_parallel_model:Model r2plus1d_test/<caffe2.python.cnn.CNNModelHelper object at 0x7f56210ad2e8> is used for testing/validation but
WARNING:data_parallel_model:has init_params=True!
WARNING:data_parallel_model:This can conflict with model training.
WARNING:data_parallel_model:Please ensure model = ModelHelper(init_params=False)
WARNING:data_parallel_model:####################################
WARNING:data_parallel_model:
INFO:data_parallel_model:Model for GPU : 1
INFO:model_helper:outputing rgb data
INFO:model_builder:creating r2plus1d, depth=34...
INFO:video_model:Number of middle filters: 144
INFO:video_model:Number of middle filters: 144
INFO:video_model:Number of middle filters: 144
INFO:video_model:Number of middle filters: 144
INFO:video_model:Number of middle filters: 144
INFO:video_model:Number of middle filters: 144
INFO:video_model:Number of middle filters: 230
INFO:video_model:Number of middle filters: 288
INFO:video_model:Number of middle filters: 288
INFO:video_model:Number of middle filters: 288
INFO:video_model:Number of middle filters: 288
INFO:video_model:Number of middle filters: 288
INFO:video_model:Number of middle filters: 288
INFO:video_model:Number of middle filters: 288
INFO:video_model:Number of middle filters: 460
INFO:video_model:Number of middle filters: 576
INFO:video_model:Number of middle filters: 576
INFO:video_model:Number of middle filters: 576
INFO:video_model:Number of middle filters: 576
INFO:video_model:Number of middle filters: 576
INFO:video_model:Number of middle filters: 576
INFO:video_model:Number of middle filters: 576
INFO:video_model:Number of middle filters: 576
INFO:video_model:Number of middle filters: 576
INFO:video_model:Number of middle filters: 576
INFO:video_model:Number of middle filters: 576
INFO:video_model:Number of middle filters: 921
INFO:video_model:Number of middle filters: 1152
INFO:video_model:Number of middle filters: 1152
INFO:video_model:Number of middle filters: 1152
INFO:video_model:Number of middle filters: 1152
INFO:video_model:Number of middle filters: 1152
INFO:data_parallel_model:Parameter update function not defined --> only forward
INFO:model_loader:copying conv1_middle_w to gpu_1/conv1_middle_w
INFO:model_loader:copying conv1_middle_spatbn_relu_s to gpu_1/conv1_middle_spatbn_relu_s
INFO:model_loader:copying conv1_middle_spatbn_relu_b to gpu_1/conv1_middle_spatbn_relu_b
INFO:model_loader:copying conv1_w to gpu_1/conv1_w
INFO:model_loader:copying conv1_spatbn_relu_s to gpu_1/conv1_spatbn_relu_s
INFO:model_loader:copying conv1_spatbn_relu_b to gpu_1/conv1_spatbn_relu_b
INFO:model_loader:copying comp_0_conv_1_middle_w to gpu_1/comp_0_conv_1_middle_w
INFO:model_loader:copying comp_0_spatbn_1_middle_s to gpu_1/comp_0_spatbn_1_middle_s
INFO:model_loader:copying comp_0_spatbn_1_middle_b to gpu_1/comp_0_spatbn_1_middle_b
INFO:model_loader:copying comp_0_conv_1_w to gpu_1/comp_0_conv_1_w
INFO:model_loader:copying comp_0_spatbn_1_s to gpu_1/comp_0_spatbn_1_s
INFO:model_loader:copying comp_0_spatbn_1_b to gpu_1/comp_0_spatbn_1_b
INFO:model_loader:copying comp_0_conv_2_middle_w to gpu_1/comp_0_conv_2_middle_w
INFO:model_loader:copying comp_0_spatbn_2_middle_s to gpu_1/comp_0_spatbn_2_middle_s
INFO:model_loader:copying comp_0_spatbn_2_middle_b to gpu_1/comp_0_spatbn_2_middle_b
INFO:model_loader:copying comp_0_conv_2_w to gpu_1/comp_0_conv_2_w
INFO:model_loader:copying comp_0_spatbn_2_s to gpu_1/comp_0_spatbn_2_s
INFO:model_loader:copying comp_0_spatbn_2_b to gpu_1/comp_0_spatbn_2_b
INFO:model_loader:copying comp_1_conv_1_middle_w to gpu_1/comp_1_conv_1_middle_w
INFO:model_loader:copying comp_1_spatbn_1_middle_s to gpu_1/comp_1_spatbn_1_middle_s
INFO:model_loader:copying comp_1_spatbn_1_middle_b to gpu_1/comp_1_spatbn_1_middle_b
INFO:model_loader:copying comp_1_conv_1_w to gpu_1/comp_1_conv_1_w
INFO:model_loader:copying comp_1_spatbn_1_s to gpu_1/comp_1_spatbn_1_s
INFO:model_loader:copying comp_1_spatbn_1_b to gpu_1/comp_1_spatbn_1_b
INFO:model_loader:copying comp_1_conv_2_middle_w to gpu_1/comp_1_conv_2_middle_w
INFO:model_loader:copying comp_1_spatbn_2_middle_s to gpu_1/comp_1_spatbn_2_middle_s
INFO:model_loader:copying comp_1_spatbn_2_middle_b to gpu_1/comp_1_spatbn_2_middle_b
INFO:model_loader:copying comp_1_conv_2_w to gpu_1/comp_1_conv_2_w
INFO:model_loader:copying comp_1_spatbn_2_s to gpu_1/comp_1_spatbn_2_s
INFO:model_loader:copying comp_1_spatbn_2_b to gpu_1/comp_1_spatbn_2_b
INFO:model_loader:found comp_2_conv_1_middle_w but blob shape do not match
INFO:model_loader:found comp_2_spatbn_1_middle_s but blob shape do not match
INFO:model_loader:found comp_2_spatbn_1_middle_b but blob shape do not match
INFO:model_loader:found comp_2_conv_1_w but blob shape do not match
INFO:model_loader:found comp_2_spatbn_1_s but blob shape do not match
INFO:model_loader:found comp_2_spatbn_1_b but blob shape do not match
INFO:model_loader:found comp_2_conv_2_middle_w but blob shape do not match
INFO:model_loader:found comp_2_spatbn_2_middle_s but blob shape do not match
INFO:model_loader:found comp_2_spatbn_2_middle_b but blob shape do not match
INFO:model_loader:found comp_2_conv_2_w but blob shape do not match
INFO:model_loader:found comp_2_spatbn_2_s but blob shape do not match
INFO:model_loader:found comp_2_spatbn_2_b but blob shape do not match
INFO:model_loader:found comp_3_conv_1_middle_w but blob shape do not match
INFO:model_loader:found comp_3_spatbn_1_middle_s but blob shape do not match
INFO:model_loader:found comp_3_spatbn_1_middle_b but blob shape do not match
INFO:model_loader:found comp_3_conv_1_w but blob shape do not match
INFO:model_loader:copying comp_3_spatbn_1_s to gpu_1/comp_3_spatbn_1_s
INFO:model_loader:copying comp_3_spatbn_1_b to gpu_1/comp_3_spatbn_1_b
INFO:model_loader:copying comp_3_conv_2_middle_w to gpu_1/comp_3_conv_2_middle_w
INFO:model_loader:copying comp_3_spatbn_2_middle_s to gpu_1/comp_3_spatbn_2_middle_s
INFO:model_loader:copying comp_3_spatbn_2_middle_b to gpu_1/comp_3_spatbn_2_middle_b
INFO:model_loader:copying comp_3_conv_2_w to gpu_1/comp_3_conv_2_w
INFO:model_loader:copying comp_3_spatbn_2_s to gpu_1/comp_3_spatbn_2_s
INFO:model_loader:copying comp_3_spatbn_2_b to gpu_1/comp_3_spatbn_2_b
INFO:model_loader:shortcut_projection_3_w not found
INFO:model_loader:shortcut_projection_3_spatbn_s not found
INFO:model_loader:shortcut_projection_3_spatbn_b not found
INFO:model_loader:found comp_4_conv_1_middle_w but blob shape do not match
INFO:model_loader:found comp_4_spatbn_1_middle_s but blob shape do not match
INFO:model_loader:found comp_4_spatbn_1_middle_b but blob shape do not match
INFO:model_loader:found comp_4_conv_1_w but blob shape do not match
INFO:model_loader:found comp_4_spatbn_1_s but blob shape do not match
INFO:model_loader:found comp_4_spatbn_1_b but blob shape do not match
INFO:model_loader:found comp_4_conv_2_middle_w but blob shape do not match
INFO:model_loader:found comp_4_spatbn_2_middle_s but blob shape do not match
INFO:model_loader:found comp_4_spatbn_2_middle_b but blob shape do not match
INFO:model_loader:found comp_4_conv_2_w but blob shape do not match
INFO:model_loader:found comp_4_spatbn_2_s but blob shape do not match
INFO:model_loader:found comp_4_spatbn_2_b but blob shape do not match
INFO:model_loader:found comp_5_conv_1_middle_w but blob shape do not match
INFO:model_loader:found comp_5_spatbn_1_middle_s but blob shape do not match
INFO:model_loader:found comp_5_spatbn_1_middle_b but blob shape do not match
INFO:model_loader:found comp_5_conv_1_w but blob shape do not match
INFO:model_loader:found comp_5_spatbn_1_s but blob shape do not match
INFO:model_loader:found comp_5_spatbn_1_b but blob shape do not match
INFO:model_loader:found comp_5_conv_2_middle_w but blob shape do not match
INFO:model_loader:found comp_5_spatbn_2_middle_s but blob shape do not match
INFO:model_loader:found comp_5_spatbn_2_middle_b but blob shape do not match
INFO:model_loader:found comp_5_conv_2_w but blob shape do not match
INFO:model_loader:found comp_5_spatbn_2_s but blob shape do not match
INFO:model_loader:found comp_5_spatbn_2_b but blob shape do not match
INFO:model_loader:found comp_6_conv_1_middle_w but blob shape do not match
INFO:model_loader:found comp_6_spatbn_1_middle_s but blob shape do not match
INFO:model_loader:found comp_6_spatbn_1_middle_b but blob shape do not match
INFO:model_loader:found comp_6_conv_1_w but blob shape do not match
INFO:model_loader:found comp_6_spatbn_1_s but blob shape do not match
INFO:model_loader:found comp_6_spatbn_1_b but blob shape do not match
INFO:model_loader:found comp_6_conv_2_middle_w but blob shape do not match
INFO:model_loader:found comp_6_spatbn_2_middle_s but blob shape do not match
INFO:model_loader:found comp_6_spatbn_2_middle_b but blob shape do not match
INFO:model_loader:found comp_6_conv_2_w but blob shape do not match
INFO:model_loader:found comp_6_spatbn_2_s but blob shape do not match
INFO:model_loader:found comp_6_spatbn_2_b but blob shape do not match
INFO:model_loader:found comp_7_conv_1_middle_w but blob shape do not match
INFO:model_loader:found comp_7_spatbn_1_middle_s but blob shape do not match
INFO:model_loader:found comp_7_spatbn_1_middle_b but blob shape do not match
INFO:model_loader:found comp_7_conv_1_w but blob shape do not match
INFO:model_loader:found comp_7_spatbn_1_s but blob shape do not match
INFO:model_loader:found comp_7_spatbn_1_b but blob shape do not match
INFO:model_loader:found comp_7_conv_2_middle_w but blob shape do not match
INFO:model_loader:found comp_7_spatbn_2_middle_s but blob shape do not match
INFO:model_loader:found comp_7_spatbn_2_middle_b but blob shape do not match
INFO:model_loader:found comp_7_conv_2_w but blob shape do not match
INFO:model_loader:found comp_7_spatbn_2_s but blob shape do not match
INFO:model_loader:found comp_7_spatbn_2_b but blob shape do not match
INFO:model_loader:shortcut_projection_7_w not found
INFO:model_loader:shortcut_projection_7_spatbn_s not found
INFO:model_loader:shortcut_projection_7_spatbn_b not found
INFO:model_loader:comp_8_conv_1_middle_w not found
INFO:model_loader:comp_8_spatbn_1_middle_s not found
INFO:model_loader:comp_8_spatbn_1_middle_b not found
INFO:model_loader:comp_8_conv_1_w not found
INFO:model_loader:comp_8_spatbn_1_s not found
INFO:model_loader:comp_8_spatbn_1_b not found
INFO:model_loader:comp_8_conv_2_middle_w not found
INFO:model_loader:comp_8_spatbn_2_middle_s not found
INFO:model_loader:comp_8_spatbn_2_middle_b not found
INFO:model_loader:comp_8_conv_2_w not found
INFO:model_loader:comp_8_spatbn_2_s not found
INFO:model_loader:comp_8_spatbn_2_b not found
INFO:model_loader:comp_9_conv_1_middle_w not found
INFO:model_loader:comp_9_spatbn_1_middle_s not found
INFO:model_loader:comp_9_spatbn_1_middle_b not found
INFO:model_loader:comp_9_conv_1_w not found
INFO:model_loader:comp_9_spatbn_1_s not found
INFO:model_loader:comp_9_spatbn_1_b not found
INFO:model_loader:comp_9_conv_2_middle_w not found
INFO:model_loader:comp_9_spatbn_2_middle_s not found
INFO:model_loader:comp_9_spatbn_2_middle_b not found
INFO:model_loader:comp_9_conv_2_w not found
INFO:model_loader:comp_9_spatbn_2_s not found
INFO:model_loader:comp_9_spatbn_2_b not found
INFO:model_loader:comp_10_conv_1_middle_w not found
INFO:model_loader:comp_10_spatbn_1_middle_s not found
INFO:model_loader:comp_10_spatbn_1_middle_b not found
INFO:model_loader:comp_10_conv_1_w not found
INFO:model_loader:comp_10_spatbn_1_s not found
INFO:model_loader:comp_10_spatbn_1_b not found
INFO:model_loader:comp_10_conv_2_middle_w not found
INFO:model_loader:comp_10_spatbn_2_middle_s not found
INFO:model_loader:comp_10_spatbn_2_middle_b not found
INFO:model_loader:comp_10_conv_2_w not found
INFO:model_loader:comp_10_spatbn_2_s not found
INFO:model_loader:comp_10_spatbn_2_b not found
INFO:model_loader:comp_11_conv_1_middle_w not found
INFO:model_loader:comp_11_spatbn_1_middle_s not found
INFO:model_loader:comp_11_spatbn_1_middle_b not found
INFO:model_loader:comp_11_conv_1_w not found
INFO:model_loader:comp_11_spatbn_1_s not found
INFO:model_loader:comp_11_spatbn_1_b not found
INFO:model_loader:comp_11_conv_2_middle_w not found
INFO:model_loader:comp_11_spatbn_2_middle_s not found
INFO:model_loader:comp_11_spatbn_2_middle_b not found
INFO:model_loader:comp_11_conv_2_w not found
INFO:model_loader:comp_11_spatbn_2_s not found
INFO:model_loader:comp_11_spatbn_2_b not found
INFO:model_loader:comp_12_conv_1_middle_w not found
INFO:model_loader:comp_12_spatbn_1_middle_s not found
INFO:model_loader:comp_12_spatbn_1_middle_b not found
INFO:model_loader:comp_12_conv_1_w not found
INFO:model_loader:comp_12_spatbn_1_s not found
INFO:model_loader:comp_12_spatbn_1_b not found
INFO:model_loader:comp_12_conv_2_middle_w not found
INFO:model_loader:comp_12_spatbn_2_middle_s not found
INFO:model_loader:comp_12_spatbn_2_middle_b not found
INFO:model_loader:comp_12_conv_2_w not found
INFO:model_loader:comp_12_spatbn_2_s not found
INFO:model_loader:comp_12_spatbn_2_b not found
INFO:model_loader:comp_13_conv_1_middle_w not found
INFO:model_loader:comp_13_spatbn_1_middle_s not found
INFO:model_loader:comp_13_spatbn_1_middle_b not found
INFO:model_loader:comp_13_conv_1_w not found
INFO:model_loader:comp_13_spatbn_1_s not found
INFO:model_loader:comp_13_spatbn_1_b not found
INFO:model_loader:comp_13_conv_2_middle_w not found
INFO:model_loader:comp_13_spatbn_2_middle_s not found
INFO:model_loader:comp_13_spatbn_2_middle_b not found
INFO:model_loader:comp_13_conv_2_w not found
INFO:model_loader:comp_13_spatbn_2_s not found
INFO:model_loader:comp_13_spatbn_2_b not found
INFO:model_loader:shortcut_projection_13_w not found
INFO:model_loader:shortcut_projection_13_spatbn_s not found
INFO:model_loader:shortcut_projection_13_spatbn_b not found
INFO:model_loader:comp_14_conv_1_middle_w not found
INFO:model_loader:comp_14_spatbn_1_middle_s not found
INFO:model_loader:comp_14_spatbn_1_middle_b not found
INFO:model_loader:comp_14_conv_1_w not found
INFO:model_loader:comp_14_spatbn_1_s not found
INFO:model_loader:comp_14_spatbn_1_b not found
INFO:model_loader:comp_14_conv_2_middle_w not found
INFO:model_loader:comp_14_spatbn_2_middle_s not found
INFO:model_loader:comp_14_spatbn_2_middle_b not found
INFO:model_loader:comp_14_conv_2_w not found
INFO:model_loader:comp_14_spatbn_2_s not found
INFO:model_loader:comp_14_spatbn_2_b not found
INFO:model_loader:comp_15_conv_1_middle_w not found
INFO:model_loader:comp_15_spatbn_1_middle_s not found
INFO:model_loader:comp_15_spatbn_1_middle_b not found
INFO:model_loader:comp_15_conv_1_w not found
INFO:model_loader:comp_15_spatbn_1_s not found
INFO:model_loader:comp_15_spatbn_1_b not found
INFO:model_loader:comp_15_conv_2_middle_w not found
INFO:model_loader:comp_15_spatbn_2_middle_s not found
INFO:model_loader:comp_15_spatbn_2_middle_b not found
INFO:model_loader:comp_15_conv_2_w not found
INFO:model_loader:comp_15_spatbn_2_s not found
INFO:model_loader:comp_15_spatbn_2_b not found
INFO:model_loader:last_out_L51_w not found
INFO:model_loader:last_out_L51_b not found
INFO:model_loader:copying conv1_middle_spatbn_relu_rm to gpu_1/conv1_middle_spatbn_relu_rm
INFO:model_loader:copying conv1_middle_spatbn_relu_riv to gpu_1/conv1_middle_spatbn_relu_riv
INFO:model_loader:copying conv1_spatbn_relu_rm to gpu_1/conv1_spatbn_relu_rm
INFO:model_loader:copying conv1_spatbn_relu_riv to gpu_1/conv1_spatbn_relu_riv
INFO:model_loader:copying comp_0_spatbn_1_middle_rm to gpu_1/comp_0_spatbn_1_middle_rm
INFO:model_loader:copying comp_0_spatbn_1_middle_riv to gpu_1/comp_0_spatbn_1_middle_riv
INFO:model_loader:copying comp_0_spatbn_1_rm to gpu_1/comp_0_spatbn_1_rm
INFO:model_loader:copying comp_0_spatbn_1_riv to gpu_1/comp_0_spatbn_1_riv
INFO:model_loader:copying comp_0_spatbn_2_middle_rm to gpu_1/comp_0_spatbn_2_middle_rm
INFO:model_loader:copying comp_0_spatbn_2_middle_riv to gpu_1/comp_0_spatbn_2_middle_riv
INFO:model_loader:copying comp_0_spatbn_2_rm to gpu_1/comp_0_spatbn_2_rm
INFO:model_loader:copying comp_0_spatbn_2_riv to gpu_1/comp_0_spatbn_2_riv
INFO:model_loader:copying comp_1_spatbn_1_middle_rm to gpu_1/comp_1_spatbn_1_middle_rm
INFO:model_loader:copying comp_1_spatbn_1_middle_riv to gpu_1/comp_1_spatbn_1_middle_riv
INFO:model_loader:copying comp_1_spatbn_1_rm to gpu_1/comp_1_spatbn_1_rm
INFO:model_loader:copying comp_1_spatbn_1_riv to gpu_1/comp_1_spatbn_1_riv
INFO:model_loader:copying comp_1_spatbn_2_middle_rm to gpu_1/comp_1_spatbn_2_middle_rm
INFO:model_loader:copying comp_1_spatbn_2_middle_riv to gpu_1/comp_1_spatbn_2_middle_riv
INFO:model_loader:copying comp_1_spatbn_2_rm to gpu_1/comp_1_spatbn_2_rm
INFO:model_loader:copying comp_1_spatbn_2_riv to gpu_1/comp_1_spatbn_2_riv
INFO:model_loader:found comp_2_spatbn_1_middle_rm but blob shape do not match
INFO:model_loader:found comp_2_spatbn_1_middle_riv but blob shape do not match
INFO:model_loader:found comp_2_spatbn_1_rm but blob shape do not match
INFO:model_loader:found comp_2_spatbn_1_riv but blob shape do not match
INFO:model_loader:found comp_2_spatbn_2_middle_rm but blob shape do not match
INFO:model_loader:found comp_2_spatbn_2_middle_riv but blob shape do not match
INFO:model_loader:found comp_2_spatbn_2_rm but blob shape do not match
INFO:model_loader:found comp_2_spatbn_2_riv but blob shape do not match
INFO:model_loader:found comp_3_spatbn_1_middle_rm but blob shape do not match
INFO:model_loader:found comp_3_spatbn_1_middle_riv but blob shape do not match
INFO:model_loader:copying comp_3_spatbn_1_rm to gpu_1/comp_3_spatbn_1_rm
INFO:model_loader:copying comp_3_spatbn_1_riv to gpu_1/comp_3_spatbn_1_riv
INFO:model_loader:copying comp_3_spatbn_2_middle_rm to gpu_1/comp_3_spatbn_2_middle_rm
INFO:model_loader:copying comp_3_spatbn_2_middle_riv to gpu_1/comp_3_spatbn_2_middle_riv
INFO:model_loader:copying comp_3_spatbn_2_rm to gpu_1/comp_3_spatbn_2_rm
INFO:model_loader:copying comp_3_spatbn_2_riv to gpu_1/comp_3_spatbn_2_riv
INFO:model_loader:shortcut_projection_3_spatbn_rm not found
INFO:model_loader:shortcut_projection_3_spatbn_riv not found
INFO:model_loader:found comp_4_spatbn_1_middle_rm but blob shape do not match
INFO:model_loader:found comp_4_spatbn_1_middle_riv but blob shape do not match
INFO:model_loader:found comp_4_spatbn_1_rm but blob shape do not match
INFO:model_loader:found comp_4_spatbn_1_riv but blob shape do not match
INFO:model_loader:found comp_4_spatbn_2_middle_rm but blob shape do not match
INFO:model_loader:found comp_4_spatbn_2_middle_riv but blob shape do not match
INFO:model_loader:found comp_4_spatbn_2_rm but blob shape do not match
INFO:model_loader:found comp_4_spatbn_2_riv but blob shape do not match
INFO:model_loader:found comp_5_spatbn_1_middle_rm but blob shape do not match
INFO:model_loader:found comp_5_spatbn_1_middle_riv but blob shape do not match
INFO:model_loader:found comp_5_spatbn_1_rm but blob shape do not match
INFO:model_loader:found comp_5_spatbn_1_riv but blob shape do not match
INFO:model_loader:found comp_5_spatbn_2_middle_rm but blob shape do not match
INFO:model_loader:found comp_5_spatbn_2_middle_riv but blob shape do not match
INFO:model_loader:found comp_5_spatbn_2_rm but blob shape do not match
INFO:model_loader:found comp_5_spatbn_2_riv but blob shape do not match
INFO:model_loader:found comp_6_spatbn_1_middle_rm but blob shape do not match
INFO:model_loader:found comp_6_spatbn_1_middle_riv but blob shape do not match
INFO:model_loader:found comp_6_spatbn_1_rm but blob shape do not match
INFO:model_loader:found comp_6_spatbn_1_riv but blob shape do not match
INFO:model_loader:found comp_6_spatbn_2_middle_rm but blob shape do not match
INFO:model_loader:found comp_6_spatbn_2_middle_riv but blob shape do not match
INFO:model_loader:found comp_6_spatbn_2_rm but blob shape do not match
INFO:model_loader:found comp_6_spatbn_2_riv but blob shape do not match
INFO:model_loader:found comp_7_spatbn_1_middle_rm but blob shape do not match
INFO:model_loader:found comp_7_spatbn_1_middle_riv but blob shape do not match
INFO:model_loader:found comp_7_spatbn_1_rm but blob shape do not match
INFO:model_loader:found comp_7_spatbn_1_riv but blob shape do not match
INFO:model_loader:found comp_7_spatbn_2_middle_rm but blob shape do not match
INFO:model_loader:found comp_7_spatbn_2_middle_riv but blob shape do not match
INFO:model_loader:found comp_7_spatbn_2_rm but blob shape do not match
INFO:model_loader:found comp_7_spatbn_2_riv but blob shape do not match
INFO:model_loader:shortcut_projection_7_spatbn_rm not found
INFO:model_loader:shortcut_projection_7_spatbn_riv not found
INFO:model_loader:comp_8_spatbn_1_middle_rm not found
INFO:model_loader:comp_8_spatbn_1_middle_riv not found
INFO:model_loader:comp_8_spatbn_1_rm not found
INFO:model_loader:comp_8_spatbn_1_riv not found
INFO:model_loader:comp_8_spatbn_2_middle_rm not found
INFO:model_loader:comp_8_spatbn_2_middle_riv not found
INFO:model_loader:comp_8_spatbn_2_rm not found
INFO:model_loader:comp_8_spatbn_2_riv not found
INFO:model_loader:comp_9_spatbn_1_middle_rm not found
INFO:model_loader:comp_9_spatbn_1_middle_riv not found
INFO:model_loader:comp_9_spatbn_1_rm not found
INFO:model_loader:comp_9_spatbn_1_riv not found
INFO:model_loader:comp_9_spatbn_2_middle_rm not found
INFO:model_loader:comp_9_spatbn_2_middle_riv not found
INFO:model_loader:comp_9_spatbn_2_rm not found
INFO:model_loader:comp_9_spatbn_2_riv not found
INFO:model_loader:comp_10_spatbn_1_middle_rm not found
INFO:model_loader:comp_10_spatbn_1_middle_riv not found
INFO:model_loader:comp_10_spatbn_1_rm not found
INFO:model_loader:comp_10_spatbn_1_riv not found
INFO:model_loader:comp_10_spatbn_2_middle_rm not found
INFO:model_loader:comp_10_spatbn_2_middle_riv not found
INFO:model_loader:comp_10_spatbn_2_rm not found
INFO:model_loader:comp_10_spatbn_2_riv not found
INFO:model_loader:comp_11_spatbn_1_middle_rm not found
INFO:model_loader:comp_11_spatbn_1_middle_riv not found
INFO:model_loader:comp_11_spatbn_1_rm not found
INFO:model_loader:comp_11_spatbn_1_riv not found
INFO:model_loader:comp_11_spatbn_2_middle_rm not found
INFO:model_loader:comp_11_spatbn_2_middle_riv not found
INFO:model_loader:comp_11_spatbn_2_rm not found
INFO:model_loader:comp_11_spatbn_2_riv not found
INFO:model_loader:comp_12_spatbn_1_middle_rm not found
INFO:model_loader:comp_12_spatbn_1_middle_riv not found
INFO:model_loader:comp_12_spatbn_1_rm not found
INFO:model_loader:comp_12_spatbn_1_riv not found
INFO:model_loader:comp_12_spatbn_2_middle_rm not found
INFO:model_loader:comp_12_spatbn_2_middle_riv not found
INFO:model_loader:comp_12_spatbn_2_rm not found
INFO:model_loader:comp_12_spatbn_2_riv not found
INFO:model_loader:comp_13_spatbn_1_middle_rm not found
INFO:model_loader:comp_13_spatbn_1_middle_riv not found
INFO:model_loader:comp_13_spatbn_1_rm not found
INFO:model_loader:comp_13_spatbn_1_riv not found
INFO:model_loader:comp_13_spatbn_2_middle_rm not found
INFO:model_loader:comp_13_spatbn_2_middle_riv not found
INFO:model_loader:comp_13_spatbn_2_rm not found
INFO:model_loader:comp_13_spatbn_2_riv not found
INFO:model_loader:shortcut_projection_13_spatbn_rm not found
INFO:model_loader:shortcut_projection_13_spatbn_riv not found
INFO:model_loader:comp_14_spatbn_1_middle_rm not found
INFO:model_loader:comp_14_spatbn_1_middle_riv not found
INFO:model_loader:comp_14_spatbn_1_rm not found
INFO:model_loader:comp_14_spatbn_1_riv not found
INFO:model_loader:comp_14_spatbn_2_middle_rm not found
INFO:model_loader:comp_14_spatbn_2_middle_riv not found
INFO:model_loader:comp_14_spatbn_2_rm not found
INFO:model_loader:comp_14_spatbn_2_riv not found
INFO:model_loader:comp_15_spatbn_1_middle_rm not found
INFO:model_loader:comp_15_spatbn_1_middle_riv not found
INFO:model_loader:comp_15_spatbn_1_rm not found
INFO:model_loader:comp_15_spatbn_1_riv not found
INFO:model_loader:comp_15_spatbn_2_middle_rm not found
INFO:model_loader:comp_15_spatbn_2_middle_riv not found
INFO:model_loader:comp_15_spatbn_2_rm not found
INFO:model_loader:comp_15_spatbn_2_riv not found
INFO:data_parallel_model:Creating checkpoint synchronization net
INFO:data_parallel_model:Run checkpoint net
INFO:train_net:Starting epoch 0/8
[NULL @ 0x7d559004d880] Failed to parse extradata
[NULL @ 0x7d559c07a480] Failed to parse extradata
[NULL @ 0x7d5594052880] Failed to parse extradata
[NULL @ 0x7d5588071480] Failed to parse extradata
[mpeg4 @ 0x7d559004d200] Video uses a non-standard and wasteful way to store B-frames ('packed B-frames'). Consider using the mpeg4_unpack_bframes bitstream filter without encoding but stream copy to fix it.
[swscaler @ 0x7d5590078cc0] Warning: data is not aligned! This can lead to a speed loss
[mpeg4 @ 0x7d559c079e00] Video uses a non-standard and wasteful way to store B-frames ('packed B-frames'). Consider using the mpeg4_unpack_bframes bitstream filter without encoding but stream copy to fix it.
[mpeg4 @ 0x7d5594052200] Video uses a non-standard and wasteful way to store B-frames ('packed B-frames'). Consider using the mpeg4_unpack_bframes bitstream filter without encoding but stream copy to fix it.
[mpeg4 @ 0x7d5588070e00] Video uses a non-standard and wasteful way to store B-frames ('packed B-frames'). Consider using the mpeg4_unpack_bframes bitstream filter without encoding but stream copy to fix it.
[NULL @ 0x7d5590064200] Failed to parse extradata
[NULL @ 0x7d558808ec80] Failed to parse extradata
[NULL @ 0x7d5594072bc0] Failed to parse extradata
[NULL @ 0x7d559c07af80] Failed to parse extradata
[mpeg4 @ 0x7d559004c080] Video uses a non-standard and wasteful way to store B-frames ('packed B-frames'). Consider using the mpeg4_unpack_bframes bitstream filter without encoding but stream copy to fix it.
[mpeg4 @ 0x7d558808fbc0] Video uses a non-standard and wasteful way to store B-frames ('packed B-frames'). Consider using the mpeg4_unpack_bframes bitstream filter without encoding but stream copy to fix it.
[mpeg4 @ 0x7d559c07a940] Video uses a non-standard and wasteful way to store B-frames ('packed B-frames'). Consider using the mpeg4_unpack_bframes bitstream filter without encoding but stream copy to fix it.
[mpeg4 @ 0x7d5594053380] Video uses a non-standard and wasteful way to store B-frames ('packed B-frames'). Consider using the mpeg4_unpack_bframes bitstream filter without encoding but stream copy to fix it.
[E net_async_base.cc:382] [enforce fail at conv_op_cudnn.cc:520] filter.dim() >= 3 && filter.dim() <= 5. 
Error from operator: 
input: "gpu_1/comp_1_sum_2" input: "gpu_1/comp_2_conv_1_middle_w" output: "gpu_1/comp_2_conv_1_middle" name: "" type: "Conv" arg { name: "kernels" ints: 1 ints: 3 ints: 3 } arg { name: "order" s: "NCHW" } arg { name: "strides" ints: 1 ints: 1 ints: 1 } arg { name: "pads" ints: 0 ints: 1 ints: 1 ints: 0 ints: 1 ints: 1 } arg { name: "exhaustive_search" i: 1 } arg { name: "ws_nbytes_limit" i: 67108864 } device_option { device_type: 1 device_id: 1 } engine: "CUDNN"frame #0: c10::ThrowEnforceNotMet(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, void const*) + 0x78 (0x7f565f770ce8 in /home/jiashu/anaconda3/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libc10.so)
frame #1: <unknown function> + 0x5a5843d (0x7f566561043d in /home/jiashu/anaconda3/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libtorch.so)
frame #2: <unknown function> + 0x5a494d8 (0x7f56656014d8 in /home/jiashu/anaconda3/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libtorch.so)
frame #3: <unknown function> + 0x2da33d0 (0x7f566295b3d0 in /home/jiashu/anaconda3/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libtorch.so)
frame #4: caffe2::AsyncNetBase::run(int, int) + 0x118 (0x7f5663facc68 in /home/jiashu/anaconda3/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libtorch.so)
frame #5: <unknown function> + 0x43fbb2a (0x7f5663fb3b2a in /home/jiashu/anaconda3/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libtorch.so)
frame #6: c10::ThreadPool::main_loop(unsigned long) + 0x2b3 (0x7f565f76b013 in /home/jiashu/anaconda3/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libc10.so)
frame #7: <unknown function> + 0xb8408 (0x7f5678d26408 in /home/jiashu/anaconda3/lib/python3.6/site-packages/google/protobuf/pyext/../../../../../libstdc++.so.6)
frame #8: <unknown function> + 0x76db (0x7f56891216db in /lib/x86_64-linux-gnu/libpthread.so.0)
frame #9: clone + 0x3f (0x7f5688e4a88f in /lib/x86_64-linux-gnu/libc.so.6)
,  op Conv
[E net_async_base.cc:134] Rethrowing exception from the run of 'r2plus1d_train'
WARNING:caffe2.python.workspace:Original python traceback for operator `34` in network `r2plus1d_train` in exception above (most recent call last):
WARNING:caffe2.python.workspace:  File "tools/train_net.py", line 501, in <module>
WARNING:caffe2.python.workspace:  File "tools/train_net.py", line 496, in main
WARNING:caffe2.python.workspace:  File "tools/train_net.py", line 280, in Train
WARNING:caffe2.python.workspace:  File "/home/jiashu/anaconda3/lib/python3.6/site-packages/caffe2/python/data_parallel_model.py", line 34, in Parallelize_GPU
WARNING:caffe2.python.workspace:  File "/home/jiashu/anaconda3/lib/python3.6/site-packages/caffe2/python/data_parallel_model.py", line 232, in Parallelize
WARNING:caffe2.python.workspace:  File "tools/train_net.py", line 217, in create_model_ops
WARNING:caffe2.python.workspace:  File "/home/jiashu/Documents/VMZ/lib/models/model_builder.py", line 128, in build_model
WARNING:caffe2.python.workspace:  File "/home/jiashu/Documents/VMZ/lib/models/r3d_model.py", line 99, in create_model
WARNING:caffe2.python.workspace:  File "/home/jiashu/Documents/VMZ/lib/models/r3d_model.py", line 188, in create_r3d
WARNING:caffe2.python.workspace:  File "/home/jiashu/Documents/VMZ/lib/models/video_model.py", line 249, in add_simple_block
WARNING:caffe2.python.workspace:  File "/home/jiashu/Documents/VMZ/lib/models/video_model.py", line 67, in add_conv
WARNING:caffe2.python.workspace:  File "/home/jiashu/anaconda3/lib/python3.6/site-packages/caffe2/python/cnn.py", line 86, in ConvNd
WARNING:caffe2.python.workspace:  File "/home/jiashu/anaconda3/lib/python3.6/site-packages/caffe2/python/brew.py", line 108, in scope_wrapper
WARNING:caffe2.python.workspace:  File "/home/jiashu/anaconda3/lib/python3.6/site-packages/caffe2/python/helpers/conv.py", line 164, in conv_nd
WARNING:caffe2.python.workspace:  File "/home/jiashu/anaconda3/lib/python3.6/site-packages/caffe2/python/helpers/conv.py", line 123, in _ConvBase
Traceback (most recent call last):
  File "tools/train_net.py", line 501, in <module>
    main()
  File "tools/train_net.py", line 496, in main
    Train(args)
  File "tools/train_net.py", line 388, in Train
    explog
  File "tools/train_net.py", line 123, in RunEpoch
    workspace.RunNet(train_model.net.Proto().name)
  File "/home/jiashu/anaconda3/lib/python3.6/site-packages/caffe2/python/workspace.py", line 254, in RunNet
    StringifyNetName(name), num_iter, allow_fail,
  File "/home/jiashu/anaconda3/lib/python3.6/site-packages/caffe2/python/workspace.py", line 215, in CallWithExceptionIntercept
    return func(*args, **kwargs)
RuntimeError: [enforce fail at conv_op_cudnn.cc:520] filter.dim() >= 3 && filter.dim() <= 5. 
Error from operator: 
input: "gpu_1/comp_1_sum_2" input: "gpu_1/comp_2_conv_1_middle_w" output: "gpu_1/comp_2_conv_1_middle" name: "" type: "Conv" arg { name: "kernels" ints: 1 ints: 3 ints: 3 } arg { name: "order" s: "NCHW" } arg { name: "strides" ints: 1 ints: 1 ints: 1 } arg { name: "pads" ints: 0 ints: 1 ints: 1 ints: 0 ints: 1 ints: 1 } arg { name: "exhaustive_search" i: 1 } arg { name: "ws_nbytes_limit" i: 67108864 } device_option { device_type: 1 device_id: 1 } engine: "CUDNN"frame #0: c10::ThrowEnforceNotMet(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, void const*) + 0x78 (0x7f565f770ce8 in /home/jiashu/anaconda3/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libc10.so)
frame #1: <unknown function> + 0x5a5843d (0x7f566561043d in /home/jiashu/anaconda3/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libtorch.so)
frame #2: <unknown function> + 0x5a494d8 (0x7f56656014d8 in /home/jiashu/anaconda3/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libtorch.so)
frame #3: <unknown function> + 0x2da33d0 (0x7f566295b3d0 in /home/jiashu/anaconda3/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libtorch.so)
frame #4: caffe2::AsyncNetBase::run(int, int) + 0x118 (0x7f5663facc68 in /home/jiashu/anaconda3/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libtorch.so)
frame #5: <unknown function> + 0x43fbb2a (0x7f5663fb3b2a in /home/jiashu/anaconda3/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libtorch.so)
frame #6: c10::ThreadPool::main_loop(unsigned long) + 0x2b3 (0x7f565f76b013 in /home/jiashu/anaconda3/lib/python3.6/site-packages/caffe2/python/../../torch/lib/libc10.so)
frame #7: <unknown function> + 0xb8408 (0x7f5678d26408 in /home/jiashu/anaconda3/lib/python3.6/site-packages/google/protobuf/pyext/../../../../../libstdc++.so.6)
frame #8: <unknown function> + 0x76db (0x7f56891216db in /lib/x86_64-linux-gnu/libpthread.so.0)
frame #9: clone + 0x3f (0x7f5688e4a88f in /lib/x86_64-linux-gnu/libc.so.6)
facebookresearch / VMZ

Finetuning HMDB51 failed #77
