When running train_net.py as of Stage 1 at Charades two-stage training in GETTING_STARTED.md, I met this error.
[INFO: misc.py: 169]: Total conv/fc/matMul FLOPs: 190.646642688(e9)
[INFO: misc.py: 170]: Total conv/fc params: 34.83232(e6)
Segmentation fault
(venv) [aac12391zj@g0034 video-long-term-feature-banks]$ Process PoolWorker-92:
Traceback (most recent call last):
File "/apps/centos7/python/2.7.18/lib/python2.7/multiprocessing/process.py", line 267, in _bootstrap
self.run()
File "/apps/centos7/python/2.7.18/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/apps/centos7/python/2.7.18/lib/python2.7/multiprocessing/pool.py", line 122, in worker
put((job, i, (False, wrapped)))
File "/apps/centos7/python/2.7.18/lib/python2.7/multiprocessing/queues.py", line 390, in put
return send(obj)
IOError: [Errno 32] Broken pipe
I used python 2.7.18, cuda 10.0.130.1, cudnn 7.6.5. And, I installed PyTorch 1.4 based on this issue.
I replaced both the GPU_NUM and BATCH_SIZE to 4 because of the limitation of our GPU setting.
I appreciate it if anyone provides any suggestion to solve this Segmentation fault and Broken pipe issue.
Thank you so much.
The detailed log is shown in the following.
(venv) [aac12391zj@g0034 video-long-term-feature-banks]$ python tools/train_net.py --config configs/charades_r50_baseline.yaml CHECKPOINT.DIR ./tmp/lfb-nl-step1-output
Ignoring @/caffe2/caffe2/contrib/gloo:gloo_ops as it is not a valid file.
Ignoring @/caffe2/caffe2/contrib/nccl:nccl_ops as it is not a valid file.
Ignoring @/caffe2/caffe2/contrib/gloo:gloo_ops_gpu as it is not a valid file.
Found Detectron ops lib: /groups/1/gac50586/migrated_from_SFA_GPFS/NISHIMURA/Charades/video-long-term-feature-banks/venv/lib/python2.7/site-packages/torch/lib/libcaffe2_detectron_ops_gpu.so
/home/aac12391zj/NISHIMURA/Charades/video-long-term-feature-banks/lib/core/config.py:427: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
yaml_config = AttrDict(yaml.load(fopen))
[INFO: config.py: 369]: Config:
[INFO: config.py: 370]: {'AVA': {'ANNOTATION_DIR': 'data/ava/annotations',
'DETECTION_SCORE_THRESH_EVAL': [0.85],
'DETECTION_SCORE_THRESH_TRAIN': 0.9,
'FORCE_TEST_FLIP': False,
'FPS': 30,
'FRAME_LIST_DIR': 'data/ava/frame_lists',
'FULL_EVAL_DURING_TRAINING': False,
'LFB_DETECTION_SCORE_THRESH': 0.9,
'LFB_MAX_NUM_FEAT_PER_STEP': 5,
'TEST_BOX_LISTS': ['ava_val_predicted_boxes.csv'],
'TEST_LFB_BOX_LISTS': ['ava_val_predicted_boxes.csv'],
'TEST_LISTS': ['val.csv'],
'TEST_MULTI_CROP': False,
'TEST_MULTI_CROP_SCALES': [224, 256, 320],
'TEST_ON_TEST_SET': False,
'TRAIN_BOX_LISTS': ['ava_train_v2.1.csv',
'ava_train_predicted_boxes.csv'],
'TRAIN_LFB_BOX_LISTS': ['ava_train_predicted_boxes.csv'],
'TRAIN_LISTS': ['train.csv'],
'TRAIN_ON_TRAIN_VAL': False},
'CHARADES': {'FPS': 24,
'FRAME_LIST_DIR': 'data/charades/frame_lists',
'LFB_CLIPS_PER_SECOND': 2,
'NUM_TEST_CLIPS_DURING_TRAINING': 9,
'NUM_TEST_CLIPS_FINAL_EVAL': 30,
'TEST_LISTS': ['val.csv'],
'TRAIN_LISTS': ['train.csv']},
'CHECKPOINT': {'CHECKPOINT_MODEL': True,
'CHECKPOINT_PERIOD': 4000,
'CONVERT_MODEL': True,
'DIR': './tmp/lfb-nl-step1-output',
'RESUME': True},
'CUDNN_WORKSPACE_LIMIT': 256,
'DATADIR': 'data/charades/frames',
'DATALOADER': {'MAX_BAD_IMAGES': 100},
'DATASET': 'charades',
'DATA_MEAN': [0.45, 0.45, 0.45],
'DATA_STD': [0.225, 0.225, 0.225],
'DEBUG': False,
'EPIC': {'ANNOTATIONS': 'EPIC_train_action_labels.csv',
'ANNOTATION_DIR': 'data/epic/annotations',
'CLASS_TYPE': '',
'FPS': 30,
'FRAME_LIST_DIR': 'data/epic/frame_lists',
'MAX_NUM_FEATS_PER_NOUN_LFB_FRAME': 10,
'NOUN_LFB_FRAMES_PER_SECOND': 1,
'TEST_LISTS': ['val.csv'],
'TRAIN_LISTS': ['train.csv'],
'VERB_LFB_CLIPS_PER_SECOND': 1},
'FBO_NL': {'DROPOUT_RATE': 0.2,
'INPUT_DROPOUT_ON': True,
'INPUT_REDUCE_DIM': True,
'LATENT_DIM': 512,
'LFB_DROPOUT_ON': True,
'NL_DROPOUT_ON': True,
'NUM_LAYERS': 2,
'PRE_ACT': True,
'PRE_ACT_LN': True,
'SCALE': True},
'GET_TRAIN_LFB': False,
'IMG_LOAD_RETRY': 10,
'INTERPOLATION': 'INTER_LINEAR',
'LFB': {'ENABLED': False,
'FBO_TYPE': 'nl',
'LFB_DIM': 2048,
'LOAD_LFB': False,
'LOAD_LFB_PATH': '',
'MODEL_PARAMS_FILE': '',
'NUM_LFB_FEAT': 500,
'WINDOW_SIZE': 100,
'WRITE_LFB': False},
'LOG_PERIOD': 10,
'MINIBATCH_QUEUE_SIZE': 64,
'MODEL': {'ALLOW_INPLACE_RELU': True,
'ALLOW_INPLACE_RESHAPE': True,
'ALLOW_INPLACE_SUM': True,
'BN_EPSILON': 1.0000001e-05,
'BN_INIT_GAMMA': 0.0,
'BN_MOMENTUM': 0.9,
'DEPTH': 50,
'DILATIONS_AFTER_CONV5': True,
'FC_INIT_STD': 0.01,
'FREEZE_BACKBONE': False,
'MEAN': 114.75,
'MEMONGER': True,
'MODEL_NAME': 'resnet_video',
'MULTI_LABEL': True,
'NUM_CLASSES': 157,
'SAMPLE_THREADS': 8,
'STD': 57.375,
'USE_AFFINE': True,
'USE_BGR': False,
'VIDEO_ARC_CHOICE': 2},
'NONLOCAL': {'BN_EPSILON': 1.0000001e-05,
'BN_INIT_GAMMA': 0.0,
'BN_MOMENTUM': 0.9,
'CONV3_NONLOCAL': True,
'CONV4_NONLOCAL': True,
'CONV_INIT_STD': 0.01,
'LAYER_MOD': 2,
'NO_BIAS': 0,
'USE_AFFINE': True,
'USE_BN': False,
'USE_MAXPOOL': True,
'USE_SCALE': True,
'USE_SOFTMAX': True,
'USE_ZERO_INIT_CONV': True},
'NUM_GPUS': 4,
'PROF_DAG': False,
'RESNETS': {'NUM_GROUPS': 1,
'STRIDE_1X1': False,
'TRANS_FUNC': 'bottleneck_transformation_3d',
'WIDTH_PER_GROUP': 64},
'RNG_SEED': 2,
'ROI': {'SCALE_FACTOR': 16, 'XFORM_RESOLUTION': 7},
'ROOT_GPU_ID': 0,
'SOLVER': {'BASE_LR': 0.02,
'GAMMA': 0.1,
'LRS': [1, 0.1],
'LR_POLICY': 'steps_with_relative_lrs',
'MAX_ITER': 24000,
'MOMENTUM': 0.9,
'NESTEROV': True,
'SCALE_MOMENTUM': True,
'SCALE_MOMENTUM_THRESHOLD': 1.1,
'STEPS': [0, 20000, 24000],
'STEP_SIZES': [20000, 4000],
'WARMUP': {'WARMUP_END_ITER': 5000,
'WARMUP_ON': False,
'WARMUP_START_LR': 0.1},
'WEIGHT_DECAY': 1.25e-05,
'WEIGHT_DECAY_BN': 0.0},
'TEST': {'BATCH_SIZE': 4,
'CROP_SHIFT': 1,
'CROP_SIZE': 256,
'DATASET_SIZE': 1814,
'DATA_TYPE': 'val',
'PARAMS_FILE': '',
'SAMPLE_RATE': 4,
'SCALE': 256,
'VIDEO_LENGTH': 32},
'TRAIN': {'BATCH_SIZE': 4,
'COMPUTE_PRECISE_BN': False,
'CROP_SIZE': 224,
'DATASET_SIZE': 7811,
'DATA_TYPE': 'train',
'DROPOUT_RATE': 0.3,
'EVAL_PERIOD': 4000,
'ITER_COMPUTE_PRECISE_BN': 200,
'JITTER_SCALES': [256, 320],
'PARAMS_FILE': 'pretrained_weights/r50_k400_pretrained.pkl',
'PCA_EIGVAL': [0.225, 0.224, 0.229],
'PCA_EIGVEC': [[-0.5675, 0.7192, 0.4009],
[-0.5808, -0.0045, -0.814],
[-0.5836, -0.6948, 0.4203]],
'RESET_START_ITER': True,
'RESUME_FROM_BATCH_SIZE': -1,
'SAMPLE_RATE': 4,
'TEST_AFTER_TRAIN': True,
'USE_COLOR_AUGMENTATION': False,
'VIDEO_LENGTH': 32},
'USE_CYTHON': False}
[E init_intrinsics_check.cc:43] CPU feature avx is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
[E init_intrinsics_check.cc:43] CPU feature avx2 is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
[E init_intrinsics_check.cc:43] CPU feature fma is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
[INFO: train_net.py: 104]: Checkpoint directory created: /groups/1/gac50586/migrated_from_SFA_GPFS/NISHIMURA/Charades/video-long-term-feature-banks/tmp/lfb-nl-step1-output/checkpoints
[WARNING: cnn.py: 25]: [====DEPRECATE WARNING====]: you are creating an object from CNNModelHelper class which will be deprecated soon. Please use ModelHelper object with brew module. For more information, please refer to caffe2.ai and python/brew.py, python/brew_test.py for more information.
[INFO: charades.py: 195]: === Charades dataset summary ===
[INFO: charades.py: 196]: Split: val
[INFO: charades.py: 197]: Use LFB? False
[INFO: charades.py: 198]: Spatial shift position: 1
[INFO: charades.py: 199]: Number of videos: 1814
[INFO: charades.py: 202]: Number of frames: 1254921
[INFO: charades_data_input.py: 51]: Creating the execution context for worker_ids: [100, 101, 102, 103], batch size: 1
[INFO: data_input_helper.py: 157]: CREATING EXECUTION CONTEXT
[INFO: data_input_helper.py: 164]: POOLS: {}
[INFO: data_input_helper.py: 165]: SHARED DATA LISTS: 0
[INFO: data_input_helper.py: 177]: worker_id: 100 list: 1
[INFO: data_input_helper.py: 179]: worker_id: 100 list keys: [100]
[INFO: data_input_helper.py: 177]: worker_id: 101 list: 2
[INFO: data_input_helper.py: 179]: worker_id: 101 list keys: [100, 101]
[INFO: data_input_helper.py: 177]: worker_id: 102 list: 3
[INFO: data_input_helper.py: 179]: worker_id: 102 list keys: [100, 101, 102]
[INFO: data_input_helper.py: 177]: worker_id: 103 list: 4
[INFO: data_input_helper.py: 179]: worker_id: 103 list keys: [100, 101, 102, 103]
[INFO: data_input_helper.py: 224]: CREATED POOL: {100: <multiprocessing.pool.Pool object at 0x2b4c57b23550>, 101: <multiprocessing.pool.Pool object at 0x2b4c57b2e850>, 102: <multiprocessing.pool.Pool object at 0x2b4c6371a990>, 103: <multiprocessing.pool.Pool object at 0x2b4c63724750>}
[INFO: data_input_helper.py: 225]: CREATED LISTS: 4
[INFO: data_input_helper.py: 226]: POOL keys: [100, 101, 102, 103]
[INFO: data_input_helper.py: 227]: LIST keys: [100, 101, 102, 103]
[INFO: data_parallel_model.py: 179]: Parallelizing model for devices: [0, 1, 2, 3]
[INFO: data_parallel_model.py: 202]: Create input and model training operators
[WARNING: data_parallel_model.py: 219]:
[WARNING: data_parallel_model.py: 220]: ############# WARNING #############
[WARNING: data_parallel_model.py: 222]: Model CNN/<models.model_builder_video.ModelBuilder object at 0x2b4c23886c90> is used for testing/validation but
[WARNING: data_parallel_model.py: 223]: has init_params=True!
[WARNING: data_parallel_model.py: 224]: This can conflict with model training.
[WARNING: data_parallel_model.py: 225]: Please ensure model = ModelHelper(init_params=False)
[WARNING: data_parallel_model.py: 226]: ####################################
[WARNING: data_parallel_model.py: 227]:
[INFO: data_parallel_model.py: 235]: Model for GPU : 0
[INFO: resnet_video.py: 150]: --------------- ResNet-50 1x64d-bottleneck_transformation_3d, charades, val, infer LFB? False, suffix: "_test" ---------------
[INFO: resnet_video.py: 166]: use_temp_convs_set: [[2], [1, 1, 1], [1, 0, 1, 0], [1, 0, 1, 0, 1, 0], [0, 1, 0]]
[INFO: resnet_video.py: 167]: temp_strides_set: [[1], [1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1, 1, 1], [1, 1, 1]]
[INFO: resnet_helper.py: 62]: res2_0 using dilation 1
[INFO: resnet_helper.py: 62]: res2_1 using dilation 1
[INFO: resnet_helper.py: 62]: res2_2 using dilation 1
[INFO: resnet_helper.py: 62]: res3_0 using dilation 1
[INFO: resnet_helper.py: 62]: res3_1 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res3_2 using dilation 1
[INFO: resnet_helper.py: 62]: res3_3 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res4_0 using dilation 1
[INFO: resnet_helper.py: 62]: res4_1 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res4_2 using dilation 1
[INFO: resnet_helper.py: 62]: res4_3 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res4_4 using dilation 1
[INFO: resnet_helper.py: 62]: res4_5 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res5_0 using dilation 2
[INFO: resnet_helper.py: 62]: res5_1 using dilation 2
[INFO: resnet_helper.py: 62]: res5_2 using dilation 2
[INFO: data_parallel_model.py: 235]: Model for GPU : 1
[INFO: resnet_video.py: 150]: --------------- ResNet-50 1x64d-bottleneck_transformation_3d, charades, val, infer LFB? False, suffix: "_test" ---------------
[INFO: resnet_video.py: 166]: use_temp_convs_set: [[2], [1, 1, 1], [1, 0, 1, 0], [1, 0, 1, 0, 1, 0], [0, 1, 0]]
[INFO: resnet_video.py: 167]: temp_strides_set: [[1], [1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1, 1, 1], [1, 1, 1]]
[INFO: resnet_helper.py: 62]: res2_0 using dilation 1
[INFO: resnet_helper.py: 62]: res2_1 using dilation 1
[INFO: resnet_helper.py: 62]: res2_2 using dilation 1
[INFO: resnet_helper.py: 62]: res3_0 using dilation 1
[INFO: resnet_helper.py: 62]: res3_1 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res3_2 using dilation 1
[INFO: resnet_helper.py: 62]: res3_3 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res4_0 using dilation 1
[INFO: resnet_helper.py: 62]: res4_1 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res4_2 using dilation 1
[INFO: resnet_helper.py: 62]: res4_3 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res4_4 using dilation 1
[INFO: resnet_helper.py: 62]: res4_5 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res5_0 using dilation 2
[INFO: resnet_helper.py: 62]: res5_1 using dilation 2
[INFO: resnet_helper.py: 62]: res5_2 using dilation 2
[INFO: data_parallel_model.py: 235]: Model for GPU : 2
[INFO: resnet_video.py: 150]: --------------- ResNet-50 1x64d-bottleneck_transformation_3d, charades, val, infer LFB? False, suffix: "_test" ---------------
[INFO: resnet_video.py: 166]: use_temp_convs_set: [[2], [1, 1, 1], [1, 0, 1, 0], [1, 0, 1, 0, 1, 0], [0, 1, 0]]
[INFO: resnet_video.py: 167]: temp_strides_set: [[1], [1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1, 1, 1], [1, 1, 1]]
[INFO: resnet_helper.py: 62]: res2_0 using dilation 1
[INFO: resnet_helper.py: 62]: res2_1 using dilation 1
[INFO: resnet_helper.py: 62]: res2_2 using dilation 1
[INFO: resnet_helper.py: 62]: res3_0 using dilation 1
[INFO: resnet_helper.py: 62]: res3_1 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res3_2 using dilation 1
[INFO: resnet_helper.py: 62]: res3_3 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res4_0 using dilation 1
[INFO: resnet_helper.py: 62]: res4_1 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res4_2 using dilation 1
[INFO: resnet_helper.py: 62]: res4_3 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res4_4 using dilation 1
[INFO: resnet_helper.py: 62]: res4_5 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res5_0 using dilation 2
[INFO: resnet_helper.py: 62]: res5_1 using dilation 2
[INFO: resnet_helper.py: 62]: res5_2 using dilation 2
[INFO: data_parallel_model.py: 235]: Model for GPU : 3
[INFO: resnet_video.py: 150]: --------------- ResNet-50 1x64d-bottleneck_transformation_3d, charades, val, infer LFB? False, suffix: "_test" ---------------
[INFO: resnet_video.py: 166]: use_temp_convs_set: [[2], [1, 1, 1], [1, 0, 1, 0], [1, 0, 1, 0, 1, 0], [0, 1, 0]]
[INFO: resnet_video.py: 167]: temp_strides_set: [[1], [1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1, 1, 1], [1, 1, 1]]
[INFO: resnet_helper.py: 62]: res2_0 using dilation 1
[INFO: resnet_helper.py: 62]: res2_1 using dilation 1
[INFO: resnet_helper.py: 62]: res2_2 using dilation 1
[INFO: resnet_helper.py: 62]: res3_0 using dilation 1
[INFO: resnet_helper.py: 62]: res3_1 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res3_2 using dilation 1
[INFO: resnet_helper.py: 62]: res3_3 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res4_0 using dilation 1
[INFO: resnet_helper.py: 62]: res4_1 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res4_2 using dilation 1
[INFO: resnet_helper.py: 62]: res4_3 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res4_4 using dilation 1
[INFO: resnet_helper.py: 62]: res4_5 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res5_0 using dilation 2
[INFO: resnet_helper.py: 62]: res5_1 using dilation 2
[INFO: resnet_helper.py: 62]: res5_2 using dilation 2
[INFO: data_parallel_model.py: 280]: Parameter update function not defined --> only forward
[I net_dag_utils.cc:102] Operator graph pruning prior to chain compute took: 0.000414924 secs
[INFO: model_builder_video.py: 160]: Starting data loader...
[INFO: dataloader.py: 366]: Pre-filling val minibatch queue
[INFO: dataloader.py: 369]: val minibatch queue pre-filled.
[INFO: metrics.py: 136]: Resetting val metrics...
[INFO: misc.py: 94]: CNN: Net proto saved to: /groups/1/gac50586/migrated_from_SFA_GPFS/NISHIMURA/Charades/video-long-term-feature-banks/tmp/lfb-nl-step1-output/CNN.pbtxt
[INFO: misc.py: 94]: CNN_init: Net proto saved to: /groups/1/gac50586/migrated_from_SFA_GPFS/NISHIMURA/Charades/video-long-term-feature-banks/tmp/lfb-nl-step1-output/CNN_init.pbtxt
[INFO: train_net.py: 124]: Test iters: 4082
[WARNING: cnn.py: 25]: [====DEPRECATE WARNING====]: you are creating an object from CNNModelHelper class which will be deprecated soon. Please use ModelHelper object with brew module. For more information, please refer to caffe2.ai and python/brew.py, python/brew_test.py for more information.
[INFO: charades.py: 195]: === Charades dataset summary ===
[INFO: charades.py: 196]: Split: train
[INFO: charades.py: 197]: Use LFB? False
[INFO: charades.py: 198]: Spatial shift position: 1
[INFO: charades.py: 199]: Number of videos: 7811
[INFO: charades.py: 202]: Number of frames: 5610191
[INFO: charades_data_input.py: 51]: Creating the execution context for worker_ids: [0, 1, 2, 3], batch size: 1
[INFO: data_input_helper.py: 157]: CREATING EXECUTION CONTEXT
[INFO: data_input_helper.py: 164]: POOLS: {100: <multiprocessing.pool.Pool object at 0x2b4c57b23550>, 101: <multiprocessing.pool.Pool object at 0x2b4c57b2e850>, 102: <multiprocessing.pool.Pool object at 0x2b4c6371a990>, 103: <multiprocessing.pool.Pool object at 0x2b4c63724750>}
[INFO: data_input_helper.py: 165]: SHARED DATA LISTS: 4
[INFO: data_input_helper.py: 177]: worker_id: 0 list: 5
[INFO: data_input_helper.py: 179]: worker_id: 0 list keys: [0, 100, 101, 102, 103]
[INFO: data_input_helper.py: 177]: worker_id: 1 list: 6
[INFO: data_input_helper.py: 179]: worker_id: 1 list keys: [0, 1, 100, 101, 102, 103]
[INFO: data_input_helper.py: 177]: worker_id: 2 list: 7
[INFO: data_input_helper.py: 179]: worker_id: 2 list keys: [0, 1, 2, 100, 101, 102, 103]
[INFO: data_input_helper.py: 177]: worker_id: 3 list: 8
[INFO: data_input_helper.py: 179]: worker_id: 3 list keys: [0, 1, 2, 3, 100, 101, 102, 103]
[INFO: data_input_helper.py: 224]: CREATED POOL: {0: <multiprocessing.pool.Pool object at 0x2b4c6392b9d0>, 1: <multiprocessing.pool.Pool object at 0x2b4e799b0510>, 2: <multiprocessing.pool.Pool object at 0x2b4e799b9510>, 3: <multiprocessing.pool.Pool object at 0x2b4e799bfed0>, 100: <multiprocessing.pool.Pool object at 0x2b4c57b23550>, 101: <multiprocessing.pool.Pool object at 0x2b4c57b2e850>, 102: <multiprocessing.pool.Pool object at 0x2b4c6371a990>, 103: <multiprocessing.pool.Pool object at 0x2b4c63724750>}
[INFO: data_input_helper.py: 225]: CREATED LISTS: 8
[INFO: data_input_helper.py: 226]: POOL keys: [0, 1, 2, 3, 100, 101, 102, 103]
[INFO: data_input_helper.py: 227]: LIST keys: [0, 1, 2, 3, 100, 101, 102, 103]
[INFO: data_parallel_model.py: 179]: Parallelizing model for devices: [0, 1, 2, 3]
[INFO: data_parallel_model.py: 202]: Create input and model training operators
[INFO: data_parallel_model.py: 235]: Model for GPU : 0
[INFO: resnet_video.py: 150]: --------------- ResNet-50 1x64d-bottleneck_transformation_3d, charades, train, infer LFB? False, suffix: "_train" ---------------
[INFO: resnet_video.py: 166]: use_temp_convs_set: [[2], [1, 1, 1], [1, 0, 1, 0], [1, 0, 1, 0, 1, 0], [0, 1, 0]]
[INFO: resnet_video.py: 167]: temp_strides_set: [[1], [1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1, 1, 1], [1, 1, 1]]
[INFO: resnet_helper.py: 62]: res2_0 using dilation 1
[INFO: resnet_helper.py: 62]: res2_1 using dilation 1
[INFO: resnet_helper.py: 62]: res2_2 using dilation 1
[INFO: resnet_helper.py: 62]: res3_0 using dilation 1
[INFO: resnet_helper.py: 62]: res3_1 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res3_2 using dilation 1
[INFO: resnet_helper.py: 62]: res3_3 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res4_0 using dilation 1
[INFO: resnet_helper.py: 62]: res4_1 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res4_2 using dilation 1
[INFO: resnet_helper.py: 62]: res4_3 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res4_4 using dilation 1
[INFO: resnet_helper.py: 62]: res4_5 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res5_0 using dilation 2
[INFO: resnet_helper.py: 62]: res5_1 using dilation 2
[INFO: resnet_helper.py: 62]: res5_2 using dilation 2
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: SigmoidCrossEntropyLoss.
[INFO: data_parallel_model.py: 235]: Model for GPU : 1
[INFO: resnet_video.py: 150]: --------------- ResNet-50 1x64d-bottleneck_transformation_3d, charades, train, infer LFB? False, suffix: "_train" ---------------
[INFO: resnet_video.py: 166]: use_temp_convs_set: [[2], [1, 1, 1], [1, 0, 1, 0], [1, 0, 1, 0, 1, 0], [0, 1, 0]]
[INFO: resnet_video.py: 167]: temp_strides_set: [[1], [1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1, 1, 1], [1, 1, 1]]
[INFO: resnet_helper.py: 62]: res2_0 using dilation 1
[INFO: resnet_helper.py: 62]: res2_1 using dilation 1
[INFO: resnet_helper.py: 62]: res2_2 using dilation 1
[INFO: resnet_helper.py: 62]: res3_0 using dilation 1
[INFO: resnet_helper.py: 62]: res3_1 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res3_2 using dilation 1
[INFO: resnet_helper.py: 62]: res3_3 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res4_0 using dilation 1
[INFO: resnet_helper.py: 62]: res4_1 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res4_2 using dilation 1
[INFO: resnet_helper.py: 62]: res4_3 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res4_4 using dilation 1
[INFO: resnet_helper.py: 62]: res4_5 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res5_0 using dilation 2
[INFO: resnet_helper.py: 62]: res5_1 using dilation 2
[INFO: resnet_helper.py: 62]: res5_2 using dilation 2
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: SigmoidCrossEntropyLoss.
[INFO: data_parallel_model.py: 235]: Model for GPU : 2
[INFO: resnet_video.py: 150]: --------------- ResNet-50 1x64d-bottleneck_transformation_3d, charades, train, infer LFB? False, suffix: "_train" ---------------
[INFO: resnet_video.py: 166]: use_temp_convs_set: [[2], [1, 1, 1], [1, 0, 1, 0], [1, 0, 1, 0, 1, 0], [0, 1, 0]]
[INFO: resnet_video.py: 167]: temp_strides_set: [[1], [1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1, 1, 1], [1, 1, 1]]
[INFO: resnet_helper.py: 62]: res2_0 using dilation 1
[INFO: resnet_helper.py: 62]: res2_1 using dilation 1
[INFO: resnet_helper.py: 62]: res2_2 using dilation 1
[INFO: resnet_helper.py: 62]: res3_0 using dilation 1
[INFO: resnet_helper.py: 62]: res3_1 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res3_2 using dilation 1
[INFO: resnet_helper.py: 62]: res3_3 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res4_0 using dilation 1
[INFO: resnet_helper.py: 62]: res4_1 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res4_2 using dilation 1
[INFO: resnet_helper.py: 62]: res4_3 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res4_4 using dilation 1
[INFO: resnet_helper.py: 62]: res4_5 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res5_0 using dilation 2
[INFO: resnet_helper.py: 62]: res5_1 using dilation 2
[INFO: resnet_helper.py: 62]: res5_2 using dilation 2
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: SigmoidCrossEntropyLoss.
[INFO: data_parallel_model.py: 235]: Model for GPU : 3
[INFO: resnet_video.py: 150]: --------------- ResNet-50 1x64d-bottleneck_transformation_3d, charades, train, infer LFB? False, suffix: "_train" ---------------
[INFO: resnet_video.py: 166]: use_temp_convs_set: [[2], [1, 1, 1], [1, 0, 1, 0], [1, 0, 1, 0, 1, 0], [0, 1, 0]]
[INFO: resnet_video.py: 167]: temp_strides_set: [[1], [1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1, 1, 1], [1, 1, 1]]
[INFO: resnet_helper.py: 62]: res2_0 using dilation 1
[INFO: resnet_helper.py: 62]: res2_1 using dilation 1
[INFO: resnet_helper.py: 62]: res2_2 using dilation 1
[INFO: resnet_helper.py: 62]: res3_0 using dilation 1
[INFO: resnet_helper.py: 62]: res3_1 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res3_2 using dilation 1
[INFO: resnet_helper.py: 62]: res3_3 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res4_0 using dilation 1
[INFO: resnet_helper.py: 62]: res4_1 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res4_2 using dilation 1
[INFO: resnet_helper.py: 62]: res4_3 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res4_4 using dilation 1
[INFO: resnet_helper.py: 62]: res4_5 using dilation 1
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py: 62]: res5_0 using dilation 2
[INFO: resnet_helper.py: 62]: res5_1 using dilation 2
[INFO: resnet_helper.py: 62]: res5_2 using dilation 2
[WARNING: model_helper.py: 442]: You are creating an op that the ModelHelper does not recognize: SigmoidCrossEntropyLoss.
[INFO: data_parallel_model.py: 269]: Adding gradient operators
[INFO: data_parallel_model.py: 313]: Add gradient all-reduces for SyncSGD
[INFO: data_parallel_model.py: 333]: Post-iteration operators for updating params
[INFO: data_parallel_model.py: 372]: Add initial parameter sync
[WARNING: data_parallel_model.py: 1737]: ------- DEPRECATED API, please use data_parallel_model.OptimizeGradientMemory() -----
[WARNING: memonger.py: 55]: NOTE: Executing memonger to optimize gradient memory
[I memonger.cc:236] Remapping 151 using 22 shared blobs.
[INFO: memonger.py: 97]: Memonger memory optimization took 0.0479099750519 secs
[WARNING: memonger.py: 55]: NOTE: Executing memonger to optimize gradient memory
[I memonger.cc:236] Remapping 151 using 22 shared blobs.
[INFO: memonger.py: 97]: Memonger memory optimization took 0.0419881343842 secs
[WARNING: memonger.py: 55]: NOTE: Executing memonger to optimize gradient memory
[I memonger.cc:236] Remapping 151 using 22 shared blobs.
[INFO: memonger.py: 97]: Memonger memory optimization took 0.0381062030792 secs
[WARNING: memonger.py: 55]: NOTE: Executing memonger to optimize gradient memory
[I memonger.cc:236] Remapping 151 using 22 shared blobs.
[INFO: memonger.py: 97]: Memonger memory optimization took 0.0379178524017 secs
[I net_dag_utils.cc:102] Operator graph pruning prior to chain compute took: 0.00156603 secs
[INFO: model_builder_video.py: 160]: Starting data loader...
[INFO: dataloader.py: 366]: Pre-filling train minibatch queue
[INFO: dataloader.py: 369]: train minibatch queue pre-filled.
[INFO: metrics.py: 136]: Resetting train metrics...
[INFO: misc.py: 94]: CNN_1: Net proto saved to: /groups/1/gac50586/migrated_from_SFA_GPFS/NISHIMURA/Charades/video-long-term-feature-banks/tmp/lfb-nl-step1-output/CNN_1.pbtxt
[INFO: misc.py: 94]: CNN_init_1: Net proto saved to: /groups/1/gac50586/migrated_from_SFA_GPFS/NISHIMURA/Charades/video-long-term-feature-banks/tmp/lfb-nl-step1-output/CNN_init_1.pbtxt
[INFO: checkpoints.py: 198]: Resume training: True
[INFO: checkpoints.py: 150]: Coverting model : pretrained_weights/r50_k400_pretrained.pkl
[INFO: checkpoints.py: 204]: Checkpoint model converted
[INFO: checkpoints.py: 208]: Initializing from pre-trained file...
[INFO: checkpoints.py: 413]: Initializing model params from file: /groups/1/gac50586/migrated_from_SFA_GPFS/NISHIMURA/Charades/video-long-term-feature-banks/tmp/lfb-nl-step1-output/checkpoints/converted_model.pkl
[INFO: checkpoints.py: 274]: Initializing model params from file: /groups/1/gac50586/migrated_from_SFA_GPFS/NISHIMURA/Charades/video-long-term-feature-banks/tmp/lfb-nl-step1-output/checkpoints/converted_model.pkl
[INFO: checkpoints.py: 368]: conv1_w loaded from weights file into: gpu_0/conv1_w (64, 3, 5, 7, 7)
[INFO: checkpoints.py: 368]: res_conv1_bn_s loaded from weights file into: gpu_0/res_conv1_bn_s (64,)
[INFO: checkpoints.py: 368]: res_conv1_bn_b loaded from weights file into: gpu_0/res_conv1_bn_b (64,)
[INFO: checkpoints.py: 368]: res2_0_branch2a_w loaded from weights file into: gpu_0/res2_0_branch2a_w (64, 64, 3, 1, 1)
[INFO: checkpoints.py: 368]: res2_0_branch2a_bn_s loaded from weights file into: gpu_0/res2_0_branch2a_bn_s (64,)
[INFO: checkpoints.py: 368]: res2_0_branch2a_bn_b loaded from weights file into: gpu_0/res2_0_branch2a_bn_b (64,)
[INFO: checkpoints.py: 368]: res2_0_branch2b_w loaded from weights file into: gpu_0/res2_0_branch2b_w (64, 64, 1, 3, 3)
[INFO: checkpoints.py: 368]: res2_0_branch2b_bn_s loaded from weights file into: gpu_0/res2_0_branch2b_bn_s (64,)
[INFO: checkpoints.py: 368]: res2_0_branch2b_bn_b loaded from weights file into: gpu_0/res2_0_branch2b_bn_b (64,)
[INFO: checkpoints.py: 368]: res2_0_branch2c_w loaded from weights file into: gpu_0/res2_0_branch2c_w (256, 64, 1, 1, 1)
[INFO: checkpoints.py: 368]: res2_0_branch2c_bn_s loaded from weights file into: gpu_0/res2_0_branch2c_bn_s (256,)
[INFO: checkpoints.py: 368]: res2_0_branch2c_bn_b loaded from weights file into: gpu_0/res2_0_branch2c_bn_b (256,)
[INFO: checkpoints.py: 368]: res2_0_branch1_w loaded from weights file into: gpu_0/res2_0_branch1_w (256, 64, 1, 1, 1)
[INFO: checkpoints.py: 368]: res2_0_branch1_bn_s loaded from weights file into: gpu_0/res2_0_branch1_bn_s (256,)
[INFO: checkpoints.py: 368]: res2_0_branch1_bn_b loaded from weights file into: gpu_0/res2_0_branch1_bn_b (256,)
[INFO: checkpoints.py: 368]: res2_1_branch2a_w loaded from weights file into: gpu_0/res2_1_branch2a_w (64, 256, 3, 1, 1)
[INFO: checkpoints.py: 368]: res2_1_branch2a_bn_s loaded from weights file into: gpu_0/res2_1_branch2a_bn_s (64,)
[INFO: checkpoints.py: 368]: res2_1_branch2a_bn_b loaded from weights file into: gpu_0/res2_1_branch2a_bn_b (64,)
[INFO: checkpoints.py: 368]: res2_1_branch2b_w loaded from weights file into: gpu_0/res2_1_branch2b_w (64, 64, 1, 3, 3)
[INFO: checkpoints.py: 368]: res2_1_branch2b_bn_s loaded from weights file into: gpu_0/res2_1_branch2b_bn_s (64,)
[INFO: checkpoints.py: 368]: res2_1_branch2b_bn_b loaded from weights file into: gpu_0/res2_1_branch2b_bn_b (64,)
[INFO: checkpoints.py: 368]: res2_1_branch2c_w loaded from weights file into: gpu_0/res2_1_branch2c_w (256, 64, 1, 1, 1)
[INFO: checkpoints.py: 368]: res2_1_branch2c_bn_s loaded from weights file into: gpu_0/res2_1_branch2c_bn_s (256,)
[INFO: checkpoints.py: 368]: res2_1_branch2c_bn_b loaded from weights file into: gpu_0/res2_1_branch2c_bn_b (256,)
[INFO: checkpoints.py: 368]: res2_2_branch2a_w loaded from weights file into: gpu_0/res2_2_branch2a_w (64, 256, 3, 1, 1)
[INFO: checkpoints.py: 368]: res2_2_branch2a_bn_s loaded from weights file into: gpu_0/res2_2_branch2a_bn_s (64,)
[INFO: checkpoints.py: 368]: res2_2_branch2a_bn_b loaded from weights file into: gpu_0/res2_2_branch2a_bn_b (64,)
[INFO: checkpoints.py: 368]: res2_2_branch2b_w loaded from weights file into: gpu_0/res2_2_branch2b_w (64, 64, 1, 3, 3)
[INFO: checkpoints.py: 368]: res2_2_branch2b_bn_s loaded from weights file into: gpu_0/res2_2_branch2b_bn_s (64,)
[INFO: checkpoints.py: 368]: res2_2_branch2b_bn_b loaded from weights file into: gpu_0/res2_2_branch2b_bn_b (64,)
[INFO: checkpoints.py: 368]: res2_2_branch2c_w loaded from weights file into: gpu_0/res2_2_branch2c_w (256, 64, 1, 1, 1)
[INFO: checkpoints.py: 368]: res2_2_branch2c_bn_s loaded from weights file into: gpu_0/res2_2_branch2c_bn_s (256,)
[INFO: checkpoints.py: 368]: res2_2_branch2c_bn_b loaded from weights file into: gpu_0/res2_2_branch2c_bn_b (256,)
[INFO: checkpoints.py: 368]: res3_0_branch2a_w loaded from weights file into: gpu_0/res3_0_branch2a_w (128, 256, 3, 1, 1)
[INFO: checkpoints.py: 368]: res3_0_branch2a_bn_s loaded from weights file into: gpu_0/res3_0_branch2a_bn_s (128,)
[INFO: checkpoints.py: 368]: res3_0_branch2a_bn_b loaded from weights file into: gpu_0/res3_0_branch2a_bn_b (128,)
[INFO: checkpoints.py: 368]: res3_0_branch2b_w loaded from weights file into: gpu_0/res3_0_branch2b_w (128, 128, 1, 3, 3)
[INFO: checkpoints.py: 368]: res3_0_branch2b_bn_s loaded from weights file into: gpu_0/res3_0_branch2b_bn_s (128,)
[INFO: checkpoints.py: 368]: res3_0_branch2b_bn_b loaded from weights file into: gpu_0/res3_0_branch2b_bn_b (128,)
[INFO: checkpoints.py: 368]: res3_0_branch2c_w loaded from weights file into: gpu_0/res3_0_branch2c_w (512, 128, 1, 1, 1)
[INFO: checkpoints.py: 368]: res3_0_branch2c_bn_s loaded from weights file into: gpu_0/res3_0_branch2c_bn_s (512,)
[INFO: checkpoints.py: 368]: res3_0_branch2c_bn_b loaded from weights file into: gpu_0/res3_0_branch2c_bn_b (512,)
[INFO: checkpoints.py: 368]: res3_0_branch1_w loaded from weights file into: gpu_0/res3_0_branch1_w (512, 256, 1, 1, 1)
[INFO: checkpoints.py: 368]: res3_0_branch1_bn_s loaded from weights file into: gpu_0/res3_0_branch1_bn_s (512,)
[INFO: checkpoints.py: 368]: res3_0_branch1_bn_b loaded from weights file into: gpu_0/res3_0_branch1_bn_b (512,)
[INFO: checkpoints.py: 368]: res3_1_branch2a_w loaded from weights file into: gpu_0/res3_1_branch2a_w (128, 512, 1, 1, 1)
[INFO: checkpoints.py: 368]: res3_1_branch2a_bn_s loaded from weights file into: gpu_0/res3_1_branch2a_bn_s (128,)
[INFO: checkpoints.py: 368]: res3_1_branch2a_bn_b loaded from weights file into: gpu_0/res3_1_branch2a_bn_b (128,)
[INFO: checkpoints.py: 368]: res3_1_branch2b_w loaded from weights file into: gpu_0/res3_1_branch2b_w (128, 128, 1, 3, 3)
[INFO: checkpoints.py: 368]: res3_1_branch2b_bn_s loaded from weights file into: gpu_0/res3_1_branch2b_bn_s (128,)
[INFO: checkpoints.py: 368]: res3_1_branch2b_bn_b loaded from weights file into: gpu_0/res3_1_branch2b_bn_b (128,)
[INFO: checkpoints.py: 368]: res3_1_branch2c_w loaded from weights file into: gpu_0/res3_1_branch2c_w (512, 128, 1, 1, 1)
[INFO: checkpoints.py: 368]: res3_1_branch2c_bn_s loaded from weights file into: gpu_0/res3_1_branch2c_bn_s (512,)
[INFO: checkpoints.py: 368]: res3_1_branch2c_bn_b loaded from weights file into: gpu_0/res3_1_branch2c_bn_b (512,)
[INFO: checkpoints.py: 368]: nonlocal_conv3_1_theta_w loaded from weights file into: gpu_0/nonlocal_conv3_1_theta_w (256, 512, 1, 1, 1)
[INFO: checkpoints.py: 368]: nonlocal_conv3_1_theta_b loaded from weights file into: gpu_0/nonlocal_conv3_1_theta_b (256,)
[INFO: checkpoints.py: 368]: nonlocal_conv3_1_phi_w loaded from weights file into: gpu_0/nonlocal_conv3_1_phi_w (256, 512, 1, 1, 1)
[INFO: checkpoints.py: 368]: nonlocal_conv3_1_phi_b loaded from weights file into: gpu_0/nonlocal_conv3_1_phi_b (256,)
[INFO: checkpoints.py: 368]: nonlocal_conv3_1_g_w loaded from weights file into: gpu_0/nonlocal_conv3_1_g_w (256, 512, 1, 1, 1)
[INFO: checkpoints.py: 368]: nonlocal_conv3_1_g_b loaded from weights file into: gpu_0/nonlocal_conv3_1_g_b (256,)
[INFO: checkpoints.py: 368]: nonlocal_conv3_1_out_w loaded from weights file into: gpu_0/nonlocal_conv3_1_out_w (512, 256, 1, 1, 1)
[INFO: checkpoints.py: 368]: nonlocal_conv3_1_out_b loaded from weights file into: gpu_0/nonlocal_conv3_1_out_b (512,)
[INFO: checkpoints.py: 368]: nonlocal_conv3_1_bn_s loaded from weights file into: gpu_0/nonlocal_conv3_1_bn_s (512,)
[INFO: checkpoints.py: 368]: nonlocal_conv3_1_bn_b loaded from weights file into: gpu_0/nonlocal_conv3_1_bn_b (512,)
[INFO: checkpoints.py: 368]: res3_2_branch2a_w loaded from weights file into: gpu_0/res3_2_branch2a_w (128, 512, 3, 1, 1)
[INFO: checkpoints.py: 368]: res3_2_branch2a_bn_s loaded from weights file into: gpu_0/res3_2_branch2a_bn_s (128,)
[INFO: checkpoints.py: 368]: res3_2_branch2a_bn_b loaded from weights file into: gpu_0/res3_2_branch2a_bn_b (128,)
[INFO: checkpoints.py: 368]: res3_2_branch2b_w loaded from weights file into: gpu_0/res3_2_branch2b_w (128, 128, 1, 3, 3)
[INFO: checkpoints.py: 368]: res3_2_branch2b_bn_s loaded from weights file into: gpu_0/res3_2_branch2b_bn_s (128,)
[INFO: checkpoints.py: 368]: res3_2_branch2b_bn_b loaded from weights file into: gpu_0/res3_2_branch2b_bn_b (128,)
[INFO: checkpoints.py: 368]: res3_2_branch2c_w loaded from weights file into: gpu_0/res3_2_branch2c_w (512, 128, 1, 1, 1)
[INFO: checkpoints.py: 368]: res3_2_branch2c_bn_s loaded from weights file into: gpu_0/res3_2_branch2c_bn_s (512,)
[INFO: checkpoints.py: 368]: res3_2_branch2c_bn_b loaded from weights file into: gpu_0/res3_2_branch2c_bn_b (512,)
[INFO: checkpoints.py: 368]: res3_3_branch2a_w loaded from weights file into: gpu_0/res3_3_branch2a_w (128, 512, 1, 1, 1)
[INFO: checkpoints.py: 368]: res3_3_branch2a_bn_s loaded from weights file into: gpu_0/res3_3_branch2a_bn_s (128,)
[INFO: checkpoints.py: 368]: res3_3_branch2a_bn_b loaded from weights file into: gpu_0/res3_3_branch2a_bn_b (128,)
[INFO: checkpoints.py: 368]: res3_3_branch2b_w loaded from weights file into: gpu_0/res3_3_branch2b_w (128, 128, 1, 3, 3)
[INFO: checkpoints.py: 368]: res3_3_branch2b_bn_s loaded from weights file into: gpu_0/res3_3_branch2b_bn_s (128,)
[INFO: checkpoints.py: 368]: res3_3_branch2b_bn_b loaded from weights file into: gpu_0/res3_3_branch2b_bn_b (128,)
[INFO: checkpoints.py: 368]: res3_3_branch2c_w loaded from weights file into: gpu_0/res3_3_branch2c_w (512, 128, 1, 1, 1)
[INFO: checkpoints.py: 368]: res3_3_branch2c_bn_s loaded from weights file into: gpu_0/res3_3_branch2c_bn_s (512,)
[INFO: checkpoints.py: 368]: res3_3_branch2c_bn_b loaded from weights file into: gpu_0/res3_3_branch2c_bn_b (512,)
[INFO: checkpoints.py: 368]: nonlocal_conv3_3_theta_w loaded from weights file into: gpu_0/nonlocal_conv3_3_theta_w (256, 512, 1, 1, 1)
[INFO: checkpoints.py: 368]: nonlocal_conv3_3_theta_b loaded from weights file into: gpu_0/nonlocal_conv3_3_theta_b (256,)
[INFO: checkpoints.py: 368]: nonlocal_conv3_3_phi_w loaded from weights file into: gpu_0/nonlocal_conv3_3_phi_w (256, 512, 1, 1, 1)
[INFO: checkpoints.py: 368]: nonlocal_conv3_3_phi_b loaded from weights file into: gpu_0/nonlocal_conv3_3_phi_b (256,)
[INFO: checkpoints.py: 368]: nonlocal_conv3_3_g_w loaded from weights file into: gpu_0/nonlocal_conv3_3_g_w (256, 512, 1, 1, 1)
[INFO: checkpoints.py: 368]: nonlocal_conv3_3_g_b loaded from weights file into: gpu_0/nonlocal_conv3_3_g_b (256,)
[INFO: checkpoints.py: 368]: nonlocal_conv3_3_out_w loaded from weights file into: gpu_0/nonlocal_conv3_3_out_w (512, 256, 1, 1, 1)
[INFO: checkpoints.py: 368]: nonlocal_conv3_3_out_b loaded from weights file into: gpu_0/nonlocal_conv3_3_out_b (512,)
[INFO: checkpoints.py: 368]: nonlocal_conv3_3_bn_s loaded from weights file into: gpu_0/nonlocal_conv3_3_bn_s (512,)
[INFO: checkpoints.py: 368]: nonlocal_conv3_3_bn_b loaded from weights file into: gpu_0/nonlocal_conv3_3_bn_b (512,)
[INFO: checkpoints.py: 368]: res4_0_branch2a_w loaded from weights file into: gpu_0/res4_0_branch2a_w (256, 512, 3, 1, 1)
[INFO: checkpoints.py: 368]: res4_0_branch2a_bn_s loaded from weights file into: gpu_0/res4_0_branch2a_bn_s (256,)
[INFO: checkpoints.py: 368]: res4_0_branch2a_bn_b loaded from weights file into: gpu_0/res4_0_branch2a_bn_b (256,)
[INFO: checkpoints.py: 368]: res4_0_branch2b_w loaded from weights file into: gpu_0/res4_0_branch2b_w (256, 256, 1, 3, 3)
[INFO: checkpoints.py: 368]: res4_0_branch2b_bn_s loaded from weights file into: gpu_0/res4_0_branch2b_bn_s (256,)
[INFO: checkpoints.py: 368]: res4_0_branch2b_bn_b loaded from weights file into: gpu_0/res4_0_branch2b_bn_b (256,)
[INFO: checkpoints.py: 368]: res4_0_branch2c_w loaded from weights file into: gpu_0/res4_0_branch2c_w (1024, 256, 1, 1, 1)
[INFO: checkpoints.py: 368]: res4_0_branch2c_bn_s loaded from weights file into: gpu_0/res4_0_branch2c_bn_s (1024,)
[INFO: checkpoints.py: 368]: res4_0_branch2c_bn_b loaded from weights file into: gpu_0/res4_0_branch2c_bn_b (1024,)
[INFO: checkpoints.py: 368]: res4_0_branch1_w loaded from weights file into: gpu_0/res4_0_branch1_w (1024, 512, 1, 1, 1)
[INFO: checkpoints.py: 368]: res4_0_branch1_bn_s loaded from weights file into: gpu_0/res4_0_branch1_bn_s (1024,)
[INFO: checkpoints.py: 368]: res4_0_branch1_bn_b loaded from weights file into: gpu_0/res4_0_branch1_bn_b (1024,)
[INFO: checkpoints.py: 368]: res4_1_branch2a_w loaded from weights file into: gpu_0/res4_1_branch2a_w (256, 1024, 1, 1, 1)
[INFO: checkpoints.py: 368]: res4_1_branch2a_bn_s loaded from weights file into: gpu_0/res4_1_branch2a_bn_s (256,)
[INFO: checkpoints.py: 368]: res4_1_branch2a_bn_b loaded from weights file into: gpu_0/res4_1_branch2a_bn_b (256,)
[INFO: checkpoints.py: 368]: res4_1_branch2b_w loaded from weights file into: gpu_0/res4_1_branch2b_w (256, 256, 1, 3, 3)
[INFO: checkpoints.py: 368]: res4_1_branch2b_bn_s loaded from weights file into: gpu_0/res4_1_branch2b_bn_s (256,)
[INFO: checkpoints.py: 368]: res4_1_branch2b_bn_b loaded from weights file into: gpu_0/res4_1_branch2b_bn_b (256,)
[INFO: checkpoints.py: 368]: res4_1_branch2c_w loaded from weights file into: gpu_0/res4_1_branch2c_w (1024, 256, 1, 1, 1)
[INFO: checkpoints.py: 368]: res4_1_branch2c_bn_s loaded from weights file into: gpu_0/res4_1_branch2c_bn_s (1024,)
[INFO: checkpoints.py: 368]: res4_1_branch2c_bn_b loaded from weights file into: gpu_0/res4_1_branch2c_bn_b (1024,)
[INFO: checkpoints.py: 368]: nonlocal_conv4_1_theta_w loaded from weights file into: gpu_0/nonlocal_conv4_1_theta_w (512, 1024, 1, 1, 1)
[INFO: checkpoints.py: 368]: nonlocal_conv4_1_theta_b loaded from weights file into: gpu_0/nonlocal_conv4_1_theta_b (512,)
[INFO: checkpoints.py: 368]: nonlocal_conv4_1_phi_w loaded from weights file into: gpu_0/nonlocal_conv4_1_phi_w (512, 1024, 1, 1, 1)
[INFO: checkpoints.py: 368]: nonlocal_conv4_1_phi_b loaded from weights file into: gpu_0/nonlocal_conv4_1_phi_b (512,)
[INFO: checkpoints.py: 368]: nonlocal_conv4_1_g_w loaded from weights file into: gpu_0/nonlocal_conv4_1_g_w (512, 1024, 1, 1, 1)
[INFO: checkpoints.py: 368]: nonlocal_conv4_1_g_b loaded from weights file into: gpu_0/nonlocal_conv4_1_g_b (512,)
[INFO: checkpoints.py: 368]: nonlocal_conv4_1_out_w loaded from weights file into: gpu_0/nonlocal_conv4_1_out_w (1024, 512, 1, 1, 1)
[INFO: checkpoints.py: 368]: nonlocal_conv4_1_out_b loaded from weights file into: gpu_0/nonlocal_conv4_1_out_b (1024,)
[INFO: checkpoints.py: 368]: nonlocal_conv4_1_bn_s loaded from weights file into: gpu_0/nonlocal_conv4_1_bn_s (1024,)
[INFO: checkpoints.py: 368]: nonlocal_conv4_1_bn_b loaded from weights file into: gpu_0/nonlocal_conv4_1_bn_b (1024,)
[INFO: checkpoints.py: 368]: res4_2_branch2a_w loaded from weights file into: gpu_0/res4_2_branch2a_w (256, 1024, 3, 1, 1)
[INFO: checkpoints.py: 368]: res4_2_branch2a_bn_s loaded from weights file into: gpu_0/res4_2_branch2a_bn_s (256,)
[INFO: checkpoints.py: 368]: res4_2_branch2a_bn_b loaded from weights file into: gpu_0/res4_2_branch2a_bn_b (256,)
[INFO: checkpoints.py: 368]: res4_2_branch2b_w loaded from weights file into: gpu_0/res4_2_branch2b_w (256, 256, 1, 3, 3)
[INFO: checkpoints.py: 368]: res4_2_branch2b_bn_s loaded from weights file into: gpu_0/res4_2_branch2b_bn_s (256,)
[INFO: checkpoints.py: 368]: res4_2_branch2b_bn_b loaded from weights file into: gpu_0/res4_2_branch2b_bn_b (256,)
[INFO: checkpoints.py: 368]: res4_2_branch2c_w loaded from weights file into: gpu_0/res4_2_branch2c_w (1024, 256, 1, 1, 1)
[INFO: checkpoints.py: 368]: res4_2_branch2c_bn_s loaded from weights file into: gpu_0/res4_2_branch2c_bn_s (1024,)
[INFO: checkpoints.py: 368]: res4_2_branch2c_bn_b loaded from weights file into: gpu_0/res4_2_branch2c_bn_b (1024,)
[INFO: checkpoints.py: 368]: res4_3_branch2a_w loaded from weights file into: gpu_0/res4_3_branch2a_w (256, 1024, 1, 1, 1)
[INFO: checkpoints.py: 368]: res4_3_branch2a_bn_s loaded from weights file into: gpu_0/res4_3_branch2a_bn_s (256,)
[INFO: checkpoints.py: 368]: res4_3_branch2a_bn_b loaded from weights file into: gpu_0/res4_3_branch2a_bn_b (256,)
[INFO: checkpoints.py: 368]: res4_3_branch2b_w loaded from weights file into: gpu_0/res4_3_branch2b_w (256, 256, 1, 3, 3)
[INFO: checkpoints.py: 368]: res4_3_branch2b_bn_s loaded from weights file into: gpu_0/res4_3_branch2b_bn_s (256,)
[INFO: checkpoints.py: 368]: res4_3_branch2b_bn_b loaded from weights file into: gpu_0/res4_3_branch2b_bn_b (256,)
[INFO: checkpoints.py: 368]: res4_3_branch2c_w loaded from weights file into: gpu_0/res4_3_branch2c_w (1024, 256, 1, 1, 1)
[INFO: checkpoints.py: 368]: res4_3_branch2c_bn_s loaded from weights file into: gpu_0/res4_3_branch2c_bn_s (1024,)
[INFO: checkpoints.py: 368]: res4_3_branch2c_bn_b loaded from weights file into: gpu_0/res4_3_branch2c_bn_b (1024,)
[INFO: checkpoints.py: 368]: nonlocal_conv4_3_theta_w loaded from weights file into: gpu_0/nonlocal_conv4_3_theta_w (512, 1024, 1, 1, 1)
[INFO: checkpoints.py: 368]: nonlocal_conv4_3_theta_b loaded from weights file into: gpu_0/nonlocal_conv4_3_theta_b (512,)
[INFO: checkpoints.py: 368]: nonlocal_conv4_3_phi_w loaded from weights file into: gpu_0/nonlocal_conv4_3_phi_w (512, 1024, 1, 1, 1)
[INFO: checkpoints.py: 368]: nonlocal_conv4_3_phi_b loaded from weights file into: gpu_0/nonlocal_conv4_3_phi_b (512,)
[INFO: checkpoints.py: 368]: nonlocal_conv4_3_g_w loaded from weights file into: gpu_0/nonlocal_conv4_3_g_w (512, 1024, 1, 1, 1)
[INFO: checkpoints.py: 368]: nonlocal_conv4_3_g_b loaded from weights file into: gpu_0/nonlocal_conv4_3_g_b (512,)
[INFO: checkpoints.py: 368]: nonlocal_conv4_3_out_w loaded from weights file into: gpu_0/nonlocal_conv4_3_out_w (1024, 512, 1, 1, 1)
[INFO: checkpoints.py: 368]: nonlocal_conv4_3_out_b loaded from weights file into: gpu_0/nonlocal_conv4_3_out_b (1024,)
[INFO: checkpoints.py: 368]: nonlocal_conv4_3_bn_s loaded from weights file into: gpu_0/nonlocal_conv4_3_bn_s (1024,)
[INFO: checkpoints.py: 368]: nonlocal_conv4_3_bn_b loaded from weights file into: gpu_0/nonlocal_conv4_3_bn_b (1024,)
[INFO: checkpoints.py: 368]: res4_4_branch2a_w loaded from weights file into: gpu_0/res4_4_branch2a_w (256, 1024, 3, 1, 1)
[INFO: checkpoints.py: 368]: res4_4_branch2a_bn_s loaded from weights file into: gpu_0/res4_4_branch2a_bn_s (256,)
[INFO: checkpoints.py: 368]: res4_4_branch2a_bn_b loaded from weights file into: gpu_0/res4_4_branch2a_bn_b (256,)
[INFO: checkpoints.py: 368]: res4_4_branch2b_w loaded from weights file into: gpu_0/res4_4_branch2b_w (256, 256, 1, 3, 3)
[INFO: checkpoints.py: 368]: res4_4_branch2b_bn_s loaded from weights file into: gpu_0/res4_4_branch2b_bn_s (256,)
[INFO: checkpoints.py: 368]: res4_4_branch2b_bn_b loaded from weights file into: gpu_0/res4_4_branch2b_bn_b (256,)
[INFO: checkpoints.py: 368]: res4_4_branch2c_w loaded from weights file into: gpu_0/res4_4_branch2c_w (1024, 256, 1, 1, 1)
[INFO: checkpoints.py: 368]: res4_4_branch2c_bn_s loaded from weights file into: gpu_0/res4_4_branch2c_bn_s (1024,)
[INFO: checkpoints.py: 368]: res4_4_branch2c_bn_b loaded from weights file into: gpu_0/res4_4_branch2c_bn_b (1024,)
[INFO: checkpoints.py: 368]: res4_5_branch2a_w loaded from weights file into: gpu_0/res4_5_branch2a_w (256, 1024, 1, 1, 1)
[INFO: checkpoints.py: 368]: res4_5_branch2a_bn_s loaded from weights file into: gpu_0/res4_5_branch2a_bn_s (256,)
[INFO: checkpoints.py: 368]: res4_5_branch2a_bn_b loaded from weights file into: gpu_0/res4_5_branch2a_bn_b (256,)
[INFO: checkpoints.py: 368]: res4_5_branch2b_w loaded from weights file into: gpu_0/res4_5_branch2b_w (256, 256, 1, 3, 3)
[INFO: checkpoints.py: 368]: res4_5_branch2b_bn_s loaded from weights file into: gpu_0/res4_5_branch2b_bn_s (256,)
[INFO: checkpoints.py: 368]: res4_5_branch2b_bn_b loaded from weights file into: gpu_0/res4_5_branch2b_bn_b (256,)
[INFO: checkpoints.py: 368]: res4_5_branch2c_w loaded from weights file into: gpu_0/res4_5_branch2c_w (1024, 256, 1, 1, 1)
[INFO: checkpoints.py: 368]: res4_5_branch2c_bn_s loaded from weights file into: gpu_0/res4_5_branch2c_bn_s (1024,)
[INFO: checkpoints.py: 368]: res4_5_branch2c_bn_b loaded from weights file into: gpu_0/res4_5_branch2c_bn_b (1024,)
[INFO: checkpoints.py: 368]: nonlocal_conv4_5_theta_w loaded from weights file into: gpu_0/nonlocal_conv4_5_theta_w (512, 1024, 1, 1, 1)
[INFO: checkpoints.py: 368]: nonlocal_conv4_5_theta_b loaded from weights file into: gpu_0/nonlocal_conv4_5_theta_b (512,)
[INFO: checkpoints.py: 368]: nonlocal_conv4_5_phi_w loaded from weights file into: gpu_0/nonlocal_conv4_5_phi_w (512, 1024, 1, 1, 1)
[INFO: checkpoints.py: 368]: nonlocal_conv4_5_phi_b loaded from weights file into: gpu_0/nonlocal_conv4_5_phi_b (512,)
[INFO: checkpoints.py: 368]: nonlocal_conv4_5_g_w loaded from weights file into: gpu_0/nonlocal_conv4_5_g_w (512, 1024, 1, 1, 1)
[INFO: checkpoints.py: 368]: nonlocal_conv4_5_g_b loaded from weights file into: gpu_0/nonlocal_conv4_5_g_b (512,)
[INFO: checkpoints.py: 368]: nonlocal_conv4_5_out_w loaded from weights file into: gpu_0/nonlocal_conv4_5_out_w (1024, 512, 1, 1, 1)
[INFO: checkpoints.py: 368]: nonlocal_conv4_5_out_b loaded from weights file into: gpu_0/nonlocal_conv4_5_out_b (1024,)
[INFO: checkpoints.py: 368]: nonlocal_conv4_5_bn_s loaded from weights file into: gpu_0/nonlocal_conv4_5_bn_s (1024,)
[INFO: checkpoints.py: 368]: nonlocal_conv4_5_bn_b loaded from weights file into: gpu_0/nonlocal_conv4_5_bn_b (1024,)
[INFO: checkpoints.py: 368]: res5_0_branch2a_w loaded from weights file into: gpu_0/res5_0_branch2a_w (512, 1024, 1, 1, 1)
[INFO: checkpoints.py: 368]: res5_0_branch2a_bn_s loaded from weights file into: gpu_0/res5_0_branch2a_bn_s (512,)
[INFO: checkpoints.py: 368]: res5_0_branch2a_bn_b loaded from weights file into: gpu_0/res5_0_branch2a_bn_b (512,)
[INFO: checkpoints.py: 368]: res5_0_branch2b_w loaded from weights file into: gpu_0/res5_0_branch2b_w (512, 512, 1, 3, 3)
[INFO: checkpoints.py: 368]: res5_0_branch2b_bn_s loaded from weights file into: gpu_0/res5_0_branch2b_bn_s (512,)
[INFO: checkpoints.py: 368]: res5_0_branch2b_bn_b loaded from weights file into: gpu_0/res5_0_branch2b_bn_b (512,)
[INFO: checkpoints.py: 368]: res5_0_branch2c_w loaded from weights file into: gpu_0/res5_0_branch2c_w (2048, 512, 1, 1, 1)
[INFO: checkpoints.py: 368]: res5_0_branch2c_bn_s loaded from weights file into: gpu_0/res5_0_branch2c_bn_s (2048,)
[INFO: checkpoints.py: 368]: res5_0_branch2c_bn_b loaded from weights file into: gpu_0/res5_0_branch2c_bn_b (2048,)
[INFO: checkpoints.py: 368]: res5_0_branch1_w loaded from weights file into: gpu_0/res5_0_branch1_w (2048, 1024, 1, 1, 1)
[INFO: checkpoints.py: 368]: res5_0_branch1_bn_s loaded from weights file into: gpu_0/res5_0_branch1_bn_s (2048,)
[INFO: checkpoints.py: 368]: res5_0_branch1_bn_b loaded from weights file into: gpu_0/res5_0_branch1_bn_b (2048,)
[INFO: checkpoints.py: 368]: res5_1_branch2a_w loaded from weights file into: gpu_0/res5_1_branch2a_w (512, 2048, 3, 1, 1)
[INFO: checkpoints.py: 368]: res5_1_branch2a_bn_s loaded from weights file into: gpu_0/res5_1_branch2a_bn_s (512,)
[INFO: checkpoints.py: 368]: res5_1_branch2a_bn_b loaded from weights file into: gpu_0/res5_1_branch2a_bn_b (512,)
[INFO: checkpoints.py: 368]: res5_1_branch2b_w loaded from weights file into: gpu_0/res5_1_branch2b_w (512, 512, 1, 3, 3)
[INFO: checkpoints.py: 368]: res5_1_branch2b_bn_s loaded from weights file into: gpu_0/res5_1_branch2b_bn_s (512,)
[INFO: checkpoints.py: 368]: res5_1_branch2b_bn_b loaded from weights file into: gpu_0/res5_1_branch2b_bn_b (512,)
[INFO: checkpoints.py: 368]: res5_1_branch2c_w loaded from weights file into: gpu_0/res5_1_branch2c_w (2048, 512, 1, 1, 1)
[INFO: checkpoints.py: 368]: res5_1_branch2c_bn_s loaded from weights file into: gpu_0/res5_1_branch2c_bn_s (2048,)
[INFO: checkpoints.py: 368]: res5_1_branch2c_bn_b loaded from weights file into: gpu_0/res5_1_branch2c_bn_b (2048,)
[INFO: checkpoints.py: 368]: res5_2_branch2a_w loaded from weights file into: gpu_0/res5_2_branch2a_w (512, 2048, 1, 1, 1)
[INFO: checkpoints.py: 368]: res5_2_branch2a_bn_s loaded from weights file into: gpu_0/res5_2_branch2a_bn_s (512,)
[INFO: checkpoints.py: 368]: res5_2_branch2a_bn_b loaded from weights file into: gpu_0/res5_2_branch2a_bn_b (512,)
[INFO: checkpoints.py: 368]: res5_2_branch2b_w loaded from weights file into: gpu_0/res5_2_branch2b_w (512, 512, 1, 3, 3)
[INFO: checkpoints.py: 368]: res5_2_branch2b_bn_s loaded from weights file into: gpu_0/res5_2_branch2b_bn_s (512,)
[INFO: checkpoints.py: 368]: res5_2_branch2b_bn_b loaded from weights file into: gpu_0/res5_2_branch2b_bn_b (512,)
[INFO: checkpoints.py: 368]: res5_2_branch2c_w loaded from weights file into: gpu_0/res5_2_branch2c_w (2048, 512, 1, 1, 1)
[INFO: checkpoints.py: 368]: res5_2_branch2c_bn_s loaded from weights file into: gpu_0/res5_2_branch2c_bn_s (2048,)
[INFO: checkpoints.py: 368]: res5_2_branch2c_bn_b loaded from weights file into: gpu_0/res5_2_branch2c_bn_b (2048,)
[INFO: checkpoints.py: 316]: pred_w not found
[INFO: checkpoints.py: 316]: pred_b not found
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/conv1_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/conv1_w
[INFO: checkpoints.py: 406]: |-> gpu_2/conv1_w
[INFO: checkpoints.py: 406]: |-> gpu_3/conv1_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res_conv1_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res_conv1_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res_conv1_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res_conv1_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res_conv1_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res_conv1_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res_conv1_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res_conv1_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_0_branch2a_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_0_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_0_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_0_branch2a_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_0_branch2a_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_0_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_0_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_0_branch2a_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_0_branch2a_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_0_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_0_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_0_branch2a_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_0_branch2b_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_0_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_0_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_0_branch2b_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_0_branch2b_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_0_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_0_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_0_branch2b_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_0_branch2b_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_0_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_0_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_0_branch2b_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_0_branch2c_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_0_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_0_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_0_branch2c_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_0_branch2c_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_0_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_0_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_0_branch2c_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_0_branch2c_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_0_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_0_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_0_branch2c_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_0_branch1_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_0_branch1_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_0_branch1_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_0_branch1_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_0_branch1_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_0_branch1_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_0_branch1_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_0_branch1_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_0_branch1_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_0_branch1_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_0_branch1_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_0_branch1_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_1_branch2a_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_1_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_1_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_1_branch2a_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_1_branch2a_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_1_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_1_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_1_branch2a_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_1_branch2a_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_1_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_1_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_1_branch2a_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_1_branch2b_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_1_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_1_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_1_branch2b_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_1_branch2b_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_1_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_1_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_1_branch2b_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_1_branch2b_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_1_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_1_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_1_branch2b_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_1_branch2c_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_1_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_1_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_1_branch2c_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_1_branch2c_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_1_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_1_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_1_branch2c_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_1_branch2c_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_1_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_1_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_1_branch2c_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_2_branch2a_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_2_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_2_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_2_branch2a_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_2_branch2a_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_2_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_2_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_2_branch2a_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_2_branch2a_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_2_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_2_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_2_branch2a_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_2_branch2b_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_2_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_2_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_2_branch2b_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_2_branch2b_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_2_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_2_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_2_branch2b_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_2_branch2b_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_2_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_2_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_2_branch2b_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_2_branch2c_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_2_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_2_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_2_branch2c_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_2_branch2c_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_2_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_2_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_2_branch2c_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_2_branch2c_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_2_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_2_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_2_branch2c_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_0_branch2a_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_0_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_0_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_0_branch2a_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_0_branch2a_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_0_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_0_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_0_branch2a_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_0_branch2a_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_0_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_0_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_0_branch2a_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_0_branch2b_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_0_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_0_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_0_branch2b_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_0_branch2b_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_0_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_0_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_0_branch2b_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_0_branch2b_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_0_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_0_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_0_branch2b_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_0_branch2c_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_0_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_0_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_0_branch2c_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_0_branch2c_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_0_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_0_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_0_branch2c_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_0_branch2c_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_0_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_0_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_0_branch2c_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_0_branch1_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_0_branch1_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_0_branch1_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_0_branch1_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_0_branch1_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_0_branch1_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_0_branch1_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_0_branch1_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_0_branch1_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_0_branch1_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_0_branch1_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_0_branch1_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_1_branch2a_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_1_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_1_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_1_branch2a_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_1_branch2a_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_1_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_1_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_1_branch2a_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_1_branch2a_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_1_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_1_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_1_branch2a_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_1_branch2b_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_1_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_1_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_1_branch2b_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_1_branch2b_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_1_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_1_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_1_branch2b_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_1_branch2b_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_1_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_1_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_1_branch2b_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_1_branch2c_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_1_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_1_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_1_branch2c_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_1_branch2c_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_1_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_1_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_1_branch2c_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_1_branch2c_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_1_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_1_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_1_branch2c_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_1_theta_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_1_theta_w
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_1_theta_w
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_1_theta_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_1_theta_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_1_theta_b
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_1_theta_b
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_1_theta_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_1_phi_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_1_phi_w
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_1_phi_w
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_1_phi_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_1_phi_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_1_phi_b
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_1_phi_b
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_1_phi_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_1_g_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_1_g_w
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_1_g_w
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_1_g_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_1_g_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_1_g_b
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_1_g_b
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_1_g_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_1_out_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_1_out_w
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_1_out_w
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_1_out_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_1_out_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_1_out_b
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_1_out_b
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_1_out_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_1_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_1_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_1_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_1_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_1_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_1_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_1_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_1_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_2_branch2a_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_2_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_2_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_2_branch2a_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_2_branch2a_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_2_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_2_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_2_branch2a_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_2_branch2a_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_2_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_2_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_2_branch2a_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_2_branch2b_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_2_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_2_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_2_branch2b_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_2_branch2b_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_2_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_2_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_2_branch2b_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_2_branch2b_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_2_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_2_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_2_branch2b_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_2_branch2c_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_2_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_2_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_2_branch2c_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_2_branch2c_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_2_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_2_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_2_branch2c_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_2_branch2c_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_2_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_2_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_2_branch2c_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_3_branch2a_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_3_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_3_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_3_branch2a_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_3_branch2a_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_3_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_3_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_3_branch2a_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_3_branch2a_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_3_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_3_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_3_branch2a_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_3_branch2b_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_3_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_3_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_3_branch2b_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_3_branch2b_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_3_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_3_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_3_branch2b_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_3_branch2b_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_3_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_3_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_3_branch2b_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_3_branch2c_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_3_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_3_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_3_branch2c_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_3_branch2c_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_3_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_3_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_3_branch2c_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_3_branch2c_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_3_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_3_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_3_branch2c_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_3_theta_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_3_theta_w
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_3_theta_w
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_3_theta_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_3_theta_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_3_theta_b
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_3_theta_b
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_3_theta_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_3_phi_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_3_phi_w
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_3_phi_w
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_3_phi_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_3_phi_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_3_phi_b
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_3_phi_b
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_3_phi_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_3_g_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_3_g_w
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_3_g_w
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_3_g_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_3_g_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_3_g_b
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_3_g_b
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_3_g_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_3_out_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_3_out_w
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_3_out_w
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_3_out_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_3_out_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_3_out_b
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_3_out_b
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_3_out_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_3_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_3_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_3_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_3_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_3_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_3_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_3_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_3_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_0_branch2a_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_0_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_0_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_0_branch2a_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_0_branch2a_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_0_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_0_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_0_branch2a_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_0_branch2a_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_0_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_0_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_0_branch2a_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_0_branch2b_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_0_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_0_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_0_branch2b_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_0_branch2b_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_0_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_0_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_0_branch2b_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_0_branch2b_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_0_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_0_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_0_branch2b_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_0_branch2c_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_0_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_0_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_0_branch2c_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_0_branch2c_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_0_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_0_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_0_branch2c_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_0_branch2c_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_0_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_0_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_0_branch2c_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_0_branch1_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_0_branch1_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_0_branch1_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_0_branch1_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_0_branch1_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_0_branch1_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_0_branch1_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_0_branch1_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_0_branch1_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_0_branch1_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_0_branch1_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_0_branch1_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_1_branch2a_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_1_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_1_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_1_branch2a_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_1_branch2a_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_1_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_1_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_1_branch2a_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_1_branch2a_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_1_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_1_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_1_branch2a_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_1_branch2b_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_1_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_1_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_1_branch2b_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_1_branch2b_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_1_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_1_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_1_branch2b_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_1_branch2b_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_1_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_1_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_1_branch2b_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_1_branch2c_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_1_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_1_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_1_branch2c_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_1_branch2c_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_1_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_1_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_1_branch2c_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_1_branch2c_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_1_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_1_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_1_branch2c_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_1_theta_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_1_theta_w
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_1_theta_w
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_1_theta_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_1_theta_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_1_theta_b
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_1_theta_b
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_1_theta_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_1_phi_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_1_phi_w
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_1_phi_w
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_1_phi_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_1_phi_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_1_phi_b
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_1_phi_b
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_1_phi_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_1_g_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_1_g_w
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_1_g_w
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_1_g_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_1_g_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_1_g_b
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_1_g_b
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_1_g_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_1_out_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_1_out_w
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_1_out_w
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_1_out_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_1_out_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_1_out_b
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_1_out_b
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_1_out_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_1_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_1_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_1_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_1_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_1_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_1_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_1_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_1_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_2_branch2a_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_2_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_2_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_2_branch2a_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_2_branch2a_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_2_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_2_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_2_branch2a_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_2_branch2a_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_2_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_2_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_2_branch2a_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_2_branch2b_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_2_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_2_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_2_branch2b_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_2_branch2b_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_2_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_2_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_2_branch2b_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_2_branch2b_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_2_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_2_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_2_branch2b_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_2_branch2c_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_2_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_2_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_2_branch2c_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_2_branch2c_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_2_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_2_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_2_branch2c_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_2_branch2c_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_2_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_2_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_2_branch2c_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_3_branch2a_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_3_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_3_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_3_branch2a_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_3_branch2a_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_3_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_3_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_3_branch2a_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_3_branch2a_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_3_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_3_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_3_branch2a_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_3_branch2b_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_3_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_3_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_3_branch2b_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_3_branch2b_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_3_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_3_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_3_branch2b_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_3_branch2b_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_3_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_3_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_3_branch2b_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_3_branch2c_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_3_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_3_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_3_branch2c_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_3_branch2c_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_3_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_3_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_3_branch2c_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_3_branch2c_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_3_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_3_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_3_branch2c_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_3_theta_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_3_theta_w
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_3_theta_w
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_3_theta_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_3_theta_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_3_theta_b
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_3_theta_b
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_3_theta_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_3_phi_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_3_phi_w
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_3_phi_w
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_3_phi_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_3_phi_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_3_phi_b
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_3_phi_b
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_3_phi_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_3_g_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_3_g_w
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_3_g_w
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_3_g_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_3_g_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_3_g_b
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_3_g_b
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_3_g_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_3_out_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_3_out_w
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_3_out_w
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_3_out_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_3_out_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_3_out_b
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_3_out_b
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_3_out_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_3_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_3_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_3_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_3_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_3_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_3_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_3_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_3_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_4_branch2a_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_4_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_4_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_4_branch2a_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_4_branch2a_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_4_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_4_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_4_branch2a_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_4_branch2a_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_4_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_4_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_4_branch2a_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_4_branch2b_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_4_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_4_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_4_branch2b_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_4_branch2b_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_4_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_4_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_4_branch2b_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_4_branch2b_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_4_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_4_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_4_branch2b_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_4_branch2c_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_4_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_4_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_4_branch2c_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_4_branch2c_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_4_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_4_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_4_branch2c_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_4_branch2c_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_4_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_4_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_4_branch2c_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_5_branch2a_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_5_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_5_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_5_branch2a_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_5_branch2a_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_5_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_5_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_5_branch2a_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_5_branch2a_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_5_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_5_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_5_branch2a_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_5_branch2b_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_5_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_5_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_5_branch2b_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_5_branch2b_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_5_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_5_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_5_branch2b_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_5_branch2b_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_5_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_5_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_5_branch2b_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_5_branch2c_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_5_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_5_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_5_branch2c_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_5_branch2c_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_5_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_5_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_5_branch2c_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_5_branch2c_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_5_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_5_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_5_branch2c_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_5_theta_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_5_theta_w
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_5_theta_w
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_5_theta_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_5_theta_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_5_theta_b
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_5_theta_b
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_5_theta_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_5_phi_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_5_phi_w
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_5_phi_w
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_5_phi_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_5_phi_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_5_phi_b
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_5_phi_b
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_5_phi_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_5_g_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_5_g_w
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_5_g_w
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_5_g_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_5_g_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_5_g_b
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_5_g_b
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_5_g_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_5_out_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_5_out_w
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_5_out_w
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_5_out_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_5_out_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_5_out_b
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_5_out_b
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_5_out_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_5_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_5_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_5_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_5_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_5_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_5_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_5_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_5_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_0_branch2a_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_0_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_0_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_0_branch2a_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_0_branch2a_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_0_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_0_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_0_branch2a_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_0_branch2a_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_0_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_0_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_0_branch2a_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_0_branch2b_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_0_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_0_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_0_branch2b_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_0_branch2b_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_0_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_0_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_0_branch2b_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_0_branch2b_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_0_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_0_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_0_branch2b_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_0_branch2c_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_0_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_0_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_0_branch2c_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_0_branch2c_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_0_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_0_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_0_branch2c_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_0_branch2c_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_0_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_0_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_0_branch2c_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_0_branch1_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_0_branch1_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_0_branch1_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_0_branch1_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_0_branch1_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_0_branch1_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_0_branch1_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_0_branch1_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_0_branch1_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_0_branch1_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_0_branch1_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_0_branch1_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_1_branch2a_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_1_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_1_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_1_branch2a_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_1_branch2a_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_1_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_1_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_1_branch2a_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_1_branch2a_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_1_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_1_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_1_branch2a_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_1_branch2b_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_1_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_1_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_1_branch2b_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_1_branch2b_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_1_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_1_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_1_branch2b_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_1_branch2b_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_1_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_1_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_1_branch2b_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_1_branch2c_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_1_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_1_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_1_branch2c_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_1_branch2c_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_1_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_1_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_1_branch2c_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_1_branch2c_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_1_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_1_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_1_branch2c_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_2_branch2a_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_2_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_2_branch2a_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_2_branch2a_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_2_branch2a_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_2_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_2_branch2a_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_2_branch2a_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_2_branch2a_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_2_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_2_branch2a_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_2_branch2a_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_2_branch2b_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_2_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_2_branch2b_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_2_branch2b_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_2_branch2b_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_2_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_2_branch2b_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_2_branch2b_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_2_branch2b_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_2_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_2_branch2b_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_2_branch2b_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_2_branch2c_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_2_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_2_branch2c_w
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_2_branch2c_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_2_branch2c_bn_s to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_2_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_2_branch2c_bn_s
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_2_branch2c_bn_s
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_2_branch2c_bn_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_2_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_2_branch2c_bn_b
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_2_branch2c_bn_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/pred_w to
[INFO: checkpoints.py: 406]: |-> gpu_1/pred_w
[INFO: checkpoints.py: 406]: |-> gpu_2/pred_w
[INFO: checkpoints.py: 406]: |-> gpu_3/pred_w
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/pred_b to
[INFO: checkpoints.py: 406]: |-> gpu_1/pred_b
[INFO: checkpoints.py: 406]: |-> gpu_2/pred_b
[INFO: checkpoints.py: 406]: |-> gpu_3/pred_b
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/conv1_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/conv1_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/conv1_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/conv1_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_0_branch2a_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_0_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_0_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_0_branch2a_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_0_branch2b_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_0_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_0_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_0_branch2b_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_0_branch2c_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_0_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_0_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_0_branch2c_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_0_branch1_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_0_branch1_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_0_branch1_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_0_branch1_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_1_branch2a_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_1_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_1_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_1_branch2a_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_1_branch2b_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_1_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_1_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_1_branch2b_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_1_branch2c_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_1_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_1_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_1_branch2c_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_2_branch2a_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_2_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_2_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_2_branch2a_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_2_branch2b_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_2_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_2_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_2_branch2b_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res2_2_branch2c_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res2_2_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res2_2_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res2_2_branch2c_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_0_branch2a_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_0_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_0_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_0_branch2a_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_0_branch2b_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_0_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_0_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_0_branch2b_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_0_branch2c_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_0_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_0_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_0_branch2c_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_0_branch1_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_0_branch1_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_0_branch1_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_0_branch1_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_1_branch2a_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_1_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_1_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_1_branch2a_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_1_branch2b_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_1_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_1_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_1_branch2b_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_1_branch2c_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_1_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_1_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_1_branch2c_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_1_theta_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_1_theta_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_1_theta_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_1_theta_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_1_theta_b_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_1_theta_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_1_theta_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_1_theta_b_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_1_phi_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_1_phi_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_1_phi_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_1_phi_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_1_phi_b_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_1_phi_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_1_phi_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_1_phi_b_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_1_g_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_1_g_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_1_g_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_1_g_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_1_g_b_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_1_g_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_1_g_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_1_g_b_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_1_out_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_1_out_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_1_out_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_1_out_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_1_out_b_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_1_out_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_1_out_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_1_out_b_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_2_branch2a_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_2_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_2_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_2_branch2a_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_2_branch2b_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_2_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_2_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_2_branch2b_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_2_branch2c_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_2_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_2_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_2_branch2c_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_3_branch2a_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_3_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_3_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_3_branch2a_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_3_branch2b_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_3_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_3_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_3_branch2b_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res3_3_branch2c_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res3_3_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res3_3_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res3_3_branch2c_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_3_theta_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_3_theta_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_3_theta_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_3_theta_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_3_theta_b_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_3_theta_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_3_theta_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_3_theta_b_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_3_phi_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_3_phi_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_3_phi_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_3_phi_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_3_phi_b_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_3_phi_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_3_phi_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_3_phi_b_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_3_g_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_3_g_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_3_g_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_3_g_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_3_g_b_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_3_g_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_3_g_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_3_g_b_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_3_out_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_3_out_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_3_out_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_3_out_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv3_3_out_b_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv3_3_out_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv3_3_out_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv3_3_out_b_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_0_branch2a_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_0_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_0_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_0_branch2a_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_0_branch2b_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_0_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_0_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_0_branch2b_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_0_branch2c_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_0_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_0_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_0_branch2c_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_0_branch1_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_0_branch1_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_0_branch1_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_0_branch1_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_1_branch2a_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_1_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_1_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_1_branch2a_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_1_branch2b_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_1_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_1_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_1_branch2b_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_1_branch2c_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_1_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_1_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_1_branch2c_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_1_theta_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_1_theta_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_1_theta_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_1_theta_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_1_theta_b_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_1_theta_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_1_theta_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_1_theta_b_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_1_phi_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_1_phi_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_1_phi_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_1_phi_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_1_phi_b_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_1_phi_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_1_phi_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_1_phi_b_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_1_g_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_1_g_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_1_g_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_1_g_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_1_g_b_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_1_g_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_1_g_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_1_g_b_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_1_out_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_1_out_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_1_out_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_1_out_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_1_out_b_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_1_out_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_1_out_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_1_out_b_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_2_branch2a_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_2_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_2_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_2_branch2a_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_2_branch2b_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_2_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_2_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_2_branch2b_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_2_branch2c_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_2_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_2_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_2_branch2c_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_3_branch2a_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_3_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_3_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_3_branch2a_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_3_branch2b_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_3_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_3_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_3_branch2b_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_3_branch2c_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_3_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_3_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_3_branch2c_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_3_theta_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_3_theta_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_3_theta_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_3_theta_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_3_theta_b_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_3_theta_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_3_theta_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_3_theta_b_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_3_phi_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_3_phi_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_3_phi_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_3_phi_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_3_phi_b_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_3_phi_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_3_phi_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_3_phi_b_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_3_g_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_3_g_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_3_g_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_3_g_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_3_g_b_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_3_g_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_3_g_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_3_g_b_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_3_out_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_3_out_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_3_out_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_3_out_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_3_out_b_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_3_out_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_3_out_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_3_out_b_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_4_branch2a_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_4_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_4_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_4_branch2a_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_4_branch2b_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_4_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_4_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_4_branch2b_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_4_branch2c_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_4_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_4_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_4_branch2c_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_5_branch2a_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_5_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_5_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_5_branch2a_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_5_branch2b_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_5_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_5_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_5_branch2b_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res4_5_branch2c_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res4_5_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res4_5_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res4_5_branch2c_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_5_theta_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_5_theta_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_5_theta_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_5_theta_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_5_theta_b_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_5_theta_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_5_theta_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_5_theta_b_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_5_phi_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_5_phi_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_5_phi_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_5_phi_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_5_phi_b_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_5_phi_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_5_phi_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_5_phi_b_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_5_g_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_5_g_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_5_g_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_5_g_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_5_g_b_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_5_g_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_5_g_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_5_g_b_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_5_out_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_5_out_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_5_out_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_5_out_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/nonlocal_conv4_5_out_b_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/nonlocal_conv4_5_out_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/nonlocal_conv4_5_out_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/nonlocal_conv4_5_out_b_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_0_branch2a_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_0_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_0_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_0_branch2a_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_0_branch2b_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_0_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_0_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_0_branch2b_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_0_branch2c_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_0_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_0_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_0_branch2c_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_0_branch1_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_0_branch1_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_0_branch1_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_0_branch1_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_1_branch2a_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_1_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_1_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_1_branch2a_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_1_branch2b_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_1_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_1_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_1_branch2b_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_1_branch2c_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_1_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_1_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_1_branch2c_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_2_branch2a_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_2_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_2_branch2a_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_2_branch2a_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_2_branch2b_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_2_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_2_branch2b_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_2_branch2b_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/res5_2_branch2c_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/res5_2_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/res5_2_branch2c_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/res5_2_branch2c_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/pred_w_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/pred_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/pred_w_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/pred_w_momentum
[INFO: checkpoints.py: 401]: Broadcasting gpu_0/pred_b_momentum to
[INFO: checkpoints.py: 406]: |-> gpu_1/pred_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_2/pred_b_momentum
[INFO: checkpoints.py: 406]: |-> gpu_3/pred_b_momentum
[INFO: checkpoints.py: 214]: Loaded: start_model_iter: 0; prev_lr: 0.00125000
[INFO: train_net.py: 143]: ------------- Training model... -------------
[INFO: metrics.py: 136]: Resetting train metrics...
[INFO: model_builder_video.py: 268]: Setting learning rate to 0.020000 at iteration 0
[INFO: model_builder_video.py: 300]: Scaling update history by 16.000000 (new/old lr)
[I net_async_base.h:207] Using specified CPU pool size: 16; device id: -1
[I net_async_base.h:212] Created new CPU pool, size: 16; device id: -1
[I cuda_nccl_gpu.cc:91] Creating NCCLContext for key: 0:0,1,2,3,
[INFO: misc.py: 114]: Printing Model: CNN_1
[INFO: misc.py: 142]: data_train : (1, 3, 32, 224, 224) => data_train : (1, 3, 32, 224, 224) ------- (op: StopGradient)
[INFO: misc.py: 142]: data_train : (1, 3, 32, 224, 224) => conv1 : (1, 64, 32, 112, 112) ------- (op: Conv)
[INFO: misc.py: 142]: conv1 : (1, 64, 32, 112, 112) => res_conv1_bn : (1, 64, 32, 112, 112) ------- (op: AffineNd)
[INFO: misc.py: 142]: res_conv1_bn : (1, 64, 32, 112, 112) => res_conv1_bn : (1, 64, 32, 112, 112) ------- (op: Relu)
[INFO: misc.py: 142]: res_conv1_bn : (1, 64, 32, 112, 112) => pool1 : (1, 64, 32, 56, 56) ------- (op: MaxPool)
[INFO: misc.py: 142]: pool1 : (1, 64, 32, 56, 56) => res2_0_branch2a : (1, 64, 32, 56, 56) ------- (op: Conv)
[INFO: misc.py: 142]: res2_0_branch2a : (1, 64, 32, 56, 56) => res2_0_branch2a_bn : (1, 64, 32, 56, 56) ------- (op: AffineNd)
[INFO: misc.py: 142]: res2_0_branch2a_bn : (1, 64, 32, 56, 56) => res2_0_branch2a_bn : (1, 64, 32, 56, 56) ------- (op: Relu)
[INFO: misc.py: 142]: res2_0_branch2a_bn : (1, 64, 32, 56, 56) => res2_0_branch2b : (1, 64, 32, 56, 56) ------- (op: Conv)
[INFO: misc.py: 142]: res2_0_branch2b : (1, 64, 32, 56, 56) => res2_0_branch2b_bn : (1, 64, 32, 56, 56) ------- (op: AffineNd)
[INFO: misc.py: 142]: res2_0_branch2b_bn : (1, 64, 32, 56, 56) => res2_0_branch2b_bn : (1, 64, 32, 56, 56) ------- (op: Relu)
[INFO: misc.py: 142]: res2_0_branch2b_bn : (1, 64, 32, 56, 56) => res2_0_branch2c : (1, 256, 32, 56, 56) ------- (op: Conv)
[INFO: misc.py: 142]: res2_0_branch2c : (1, 256, 32, 56, 56) => res2_0_branch2c_bn : (1, 256, 32, 56, 56) ------- (op: AffineNd)
[INFO: misc.py: 142]: pool1 : (1, 64, 32, 56, 56) => res2_0_branch1 : (1, 256, 32, 56, 56) ------- (op: Conv)
[INFO: misc.py: 142]: res2_0_branch1 : (1, 256, 32, 56, 56) => res2_0_branch1_bn : (1, 256, 32, 56, 56) ------- (op: AffineNd)
[INFO: misc.py: 142]: res2_0_branch2c_bn : (1, 256, 32, 56, 56) => res2_0_branch2c_bn : (1, 256, 32, 56, 56) ------- (op: Sum)
[INFO: misc.py: 142]: res2_0_branch1_bn : (1, 256, 32, 56, 56) => res2_0_branch2c_bn : (1, 256, 32, 56, 56) ------|
[INFO: misc.py: 142]: res2_0_branch2c_bn : (1, 256, 32, 56, 56) => res2_0_branch2c_bn : (1, 256, 32, 56, 56) ------- (op: Relu)
[INFO: misc.py: 142]: res2_0_branch2c_bn : (1, 256, 32, 56, 56) => res2_1_branch2a : (1, 64, 32, 56, 56) ------- (op: Conv)
[INFO: misc.py: 142]: res2_1_branch2a : (1, 64, 32, 56, 56) => res2_1_branch2a_bn : (1, 64, 32, 56, 56) ------- (op: AffineNd)
[INFO: misc.py: 142]: res2_1_branch2a_bn : (1, 64, 32, 56, 56) => res2_1_branch2a_bn : (1, 64, 32, 56, 56) ------- (op: Relu)
[INFO: misc.py: 142]: res2_1_branch2a_bn : (1, 64, 32, 56, 56) => res2_1_branch2b : (1, 64, 32, 56, 56) ------- (op: Conv)
[INFO: misc.py: 142]: res2_1_branch2b : (1, 64, 32, 56, 56) => res2_1_branch2b_bn : (1, 64, 32, 56, 56) ------- (op: AffineNd)
[INFO: misc.py: 142]: res2_1_branch2b_bn : (1, 64, 32, 56, 56) => res2_1_branch2b_bn : (1, 64, 32, 56, 56) ------- (op: Relu)
[INFO: misc.py: 142]: res2_1_branch2b_bn : (1, 64, 32, 56, 56) => res2_1_branch2c : (1, 256, 32, 56, 56) ------- (op: Conv)
[INFO: misc.py: 142]: res2_1_branch2c : (1, 256, 32, 56, 56) => res2_1_branch2c_bn : (1, 256, 32, 56, 56) ------- (op: AffineNd)
[INFO: misc.py: 142]: res2_1_branch2c_bn : (1, 256, 32, 56, 56) => res2_1_branch2c_bn : (1, 256, 32, 56, 56) ------- (op: Sum)
[INFO: misc.py: 142]: res2_0_branch2c_bn : (1, 256, 32, 56, 56) => res2_1_branch2c_bn : (1, 256, 32, 56, 56) ------|
[INFO: misc.py: 142]: res2_1_branch2c_bn : (1, 256, 32, 56, 56) => res2_1_branch2c_bn : (1, 256, 32, 56, 56) ------- (op: Relu)
[INFO: misc.py: 142]: res2_1_branch2c_bn : (1, 256, 32, 56, 56) => res2_2_branch2a : (1, 64, 32, 56, 56) ------- (op: Conv)
[INFO: misc.py: 142]: res2_2_branch2a : (1, 64, 32, 56, 56) => res2_2_branch2a_bn : (1, 64, 32, 56, 56) ------- (op: AffineNd)
[INFO: misc.py: 142]: res2_2_branch2a_bn : (1, 64, 32, 56, 56) => res2_2_branch2a_bn : (1, 64, 32, 56, 56) ------- (op: Relu)
[INFO: misc.py: 142]: res2_2_branch2a_bn : (1, 64, 32, 56, 56) => res2_2_branch2b : (1, 64, 32, 56, 56) ------- (op: Conv)
[INFO: misc.py: 142]: res2_2_branch2b : (1, 64, 32, 56, 56) => res2_2_branch2b_bn : (1, 64, 32, 56, 56) ------- (op: AffineNd)
[INFO: misc.py: 142]: res2_2_branch2b_bn : (1, 64, 32, 56, 56) => res2_2_branch2b_bn : (1, 64, 32, 56, 56) ------- (op: Relu)
[INFO: misc.py: 142]: res2_2_branch2b_bn : (1, 64, 32, 56, 56) => res2_2_branch2c : (1, 256, 32, 56, 56) ------- (op: Conv)
[INFO: misc.py: 142]: res2_2_branch2c : (1, 256, 32, 56, 56) => res2_2_branch2c_bn : (1, 256, 32, 56, 56) ------- (op: AffineNd)
[INFO: misc.py: 142]: res2_2_branch2c_bn : (1, 256, 32, 56, 56) => res2_2_branch2c_bn : (1, 256, 32, 56, 56) ------- (op: Sum)
[INFO: misc.py: 142]: res2_1_branch2c_bn : (1, 256, 32, 56, 56) => res2_2_branch2c_bn : (1, 256, 32, 56, 56) ------|
[INFO: misc.py: 142]: res2_2_branch2c_bn : (1, 256, 32, 56, 56) => res2_2_branch2c_bn : (1, 256, 32, 56, 56) ------- (op: Relu)
[INFO: misc.py: 142]: res2_2_branch2c_bn : (1, 256, 32, 56, 56) => pool2 : (1, 256, 16, 56, 56) ------- (op: MaxPool)
[INFO: misc.py: 142]: pool2 : (1, 256, 16, 56, 56) => res3_0_branch2a : (1, 128, 16, 56, 56) ------- (op: Conv)
[INFO: misc.py: 142]: res3_0_branch2a : (1, 128, 16, 56, 56) => res3_0_branch2a_bn : (1, 128, 16, 56, 56) ------- (op: AffineNd)
[INFO: misc.py: 142]: res3_0_branch2a_bn : (1, 128, 16, 56, 56) => res3_0_branch2a_bn : (1, 128, 16, 56, 56) ------- (op: Relu)
[INFO: misc.py: 142]: res3_0_branch2a_bn : (1, 128, 16, 56, 56) => res3_0_branch2b : (1, 128, 16, 28, 28) ------- (op: Conv)
[INFO: misc.py: 142]: res3_0_branch2b : (1, 128, 16, 28, 28) => res3_0_branch2b_bn : (1, 128, 16, 28, 28) ------- (op: AffineNd)
[INFO: misc.py: 142]: res3_0_branch2b_bn : (1, 128, 16, 28, 28) => res3_0_branch2b_bn : (1, 128, 16, 28, 28) ------- (op: Relu)
[INFO: misc.py: 142]: res3_0_branch2b_bn : (1, 128, 16, 28, 28) => res3_0_branch2c : (1, 512, 16, 28, 28) ------- (op: Conv)
[INFO: misc.py: 142]: res3_0_branch2c : (1, 512, 16, 28, 28) => res3_0_branch2c_bn : (1, 512, 16, 28, 28) ------- (op: AffineNd)
[INFO: misc.py: 142]: pool2 : (1, 256, 16, 56, 56) => res3_0_branch1 : (1, 512, 16, 28, 28) ------- (op: Conv)
[INFO: misc.py: 142]: res3_0_branch1 : (1, 512, 16, 28, 28) => res3_0_branch1_bn : (1, 512, 16, 28, 28) ------- (op: AffineNd)
[INFO: misc.py: 142]: res3_0_branch2c_bn : (1, 512, 16, 28, 28) => res3_0_branch2c_bn : (1, 512, 16, 28, 28) ------- (op: Sum)
[INFO: misc.py: 142]: res3_0_branch1_bn : (1, 512, 16, 28, 28) => res3_0_branch2c_bn : (1, 512, 16, 28, 28) ------|
[INFO: misc.py: 142]: res3_0_branch2c_bn : (1, 512, 16, 28, 28) => res3_0_branch2c_bn : (1, 512, 16, 28, 28) ------- (op: Relu)
[INFO: misc.py: 142]: res3_0_branch2c_bn : (1, 512, 16, 28, 28) => res3_1_branch2a : (1, 128, 16, 28, 28) ------- (op: Conv)
[INFO: misc.py: 142]: res3_1_branch2a : (1, 128, 16, 28, 28) => res3_1_branch2a_bn : (1, 128, 16, 28, 28) ------- (op: AffineNd)
[INFO: misc.py: 142]: res3_1_branch2a_bn : (1, 128, 16, 28, 28) => res3_1_branch2a_bn : (1, 128, 16, 28, 28) ------- (op: Relu)
[INFO: misc.py: 142]: res3_1_branch2a_bn : (1, 128, 16, 28, 28) => res3_1_branch2b : (1, 128, 16, 28, 28) ------- (op: Conv)
[INFO: misc.py: 142]: res3_1_branch2b : (1, 128, 16, 28, 28) => res3_1_branch2b_bn : (1, 128, 16, 28, 28) ------- (op: AffineNd)
[INFO: misc.py: 142]: res3_1_branch2b_bn : (1, 128, 16, 28, 28) => res3_1_branch2b_bn : (1, 128, 16, 28, 28) ------- (op: Relu)
[INFO: misc.py: 142]: res3_1_branch2b_bn : (1, 128, 16, 28, 28) => res3_1_branch2c : (1, 512, 16, 28, 28) ------- (op: Conv)
[INFO: misc.py: 142]: res3_1_branch2c : (1, 512, 16, 28, 28) => res3_1_branch2c_bn : (1, 512, 16, 28, 28) ------- (op: AffineNd)
[INFO: misc.py: 142]: res3_1_branch2c_bn : (1, 512, 16, 28, 28) => res3_1_branch2c_bn : (1, 512, 16, 28, 28) ------- (op: Sum)
[INFO: misc.py: 142]: res3_0_branch2c_bn : (1, 512, 16, 28, 28) => res3_1_branch2c_bn : (1, 512, 16, 28, 28) ------|
[INFO: misc.py: 142]: res3_1_branch2c_bn : (1, 512, 16, 28, 28) => res3_1_branch2c_bn : (1, 512, 16, 28, 28) ------- (op: Relu)
[INFO: misc.py: 142]: res3_1_branch2c_bn : (1, 512, 16, 28, 28) => res3_1_branch2c_bn_trans : (4, 4, 512, 28, 28) ------- (op: Transpose)
[INFO: misc.py: 142]: res3_1_branch2c_bn_trans : (4, 4, 512, 28, 28) => res3_1_branch2c_bn_trans : (4, 4, 512, 28, 28) ------- (op: Reshape)
[INFO: misc.py: 142]: res3_1_branch2c_bn_trans : (4, 4, 512, 28, 28) => res3_1_branch2c_bn_trans_trans : (4, 512, 4, 28, 28) ------- (op: Transpose)
[INFO: misc.py: 142]: res3_1_branch2c_bn_trans_trans: (4, 512, 4, 28, 28) => nonlocal_conv3_1_theta : (4, 256, 3136) ------- (op: Conv)
[INFO: misc.py: 142]: res3_1_branch2c_bn_trans_trans: (4, 512, 4, 28, 28) => nonlocal_conv3_1_pool : (4, 512, 4, 14, 14) ------- (op: MaxPool)
[INFO: misc.py: 142]: nonlocal_conv3_1_pool : (4, 512, 4, 14, 14) => nonlocal_conv3_1_phi : (4, 256, 784) ------- (op: Conv)
[INFO: misc.py: 142]: nonlocal_conv3_1_pool : (4, 512, 4, 14, 14) => nonlocal_conv3_1_g : (4, 256, 784) ------- (op: Conv)
[INFO: misc.py: 142]: nonlocal_conv3_1_theta : (4, 256, 3136) => nonlocal_conv3_1_theta : (4, 256, 3136) ------- (op: Reshape)
[INFO: misc.py: 142]: nonlocal_conv3_1_phi : (4, 256, 784) => nonlocal_conv3_1_phi : (4, 256, 784) ------- (op: Reshape)
[INFO: misc.py: 142]: nonlocal_conv3_1_g : (4, 256, 784) => nonlocal_conv3_1_g : (4, 256, 784) ------- (op: Reshape)
[INFO: misc.py: 142]: nonlocal_conv3_1_theta : (4, 256, 3136) => nonlocal_conv3_1_affinity : (4, 3136, 784) ------- (op: BatchMatMul)
[INFO: misc.py: 142]: nonlocal_conv3_1_phi : (4, 256, 784) => nonlocal_conv3_1_affinity : (4, 3136, 784) ------|
[INFO: misc.py: 142]: nonlocal_conv3_1_affinity : (4, 3136, 784) => nonlocal_conv3_1_affinity : (4, 3136, 784) ------- (op: Scale)
[INFO: misc.py: 142]: nonlocal_conv3_1_affinity : (4, 3136, 784) => nonlocal_conv3_1_affinity_prob : (4, 3136, 784) ------- (op: Softmax)
[INFO: misc.py: 142]: nonlocal_conv3_1_g : (4, 256, 784) => nonlocal_conv3_1_y : (4, 256, 4, 28, 28) ------- (op: BatchMatMul)
[INFO: misc.py: 142]: nonlocal_conv3_1_affinity_prob: (4, 3136, 784) => nonlocal_conv3_1_y : (4, 256, 4, 28, 28) ------|
[INFO: misc.py: 142]: nonlocal_conv3_1_y : (4, 256, 4, 28, 28) => nonlocal_conv3_1_y : (4, 256, 4, 28, 28) ------- (op: Reshape)
[INFO: misc.py: 142]: nonlocal_conv3_1_theta_shape5d: (5,) => nonlocal_conv3_1_y : (4, 256, 4, 28, 28) ------|
[INFO: misc.py: 142]: nonlocal_conv3_1_y : (4, 256, 4, 28, 28) => nonlocal_conv3_1_out : (4, 512, 4, 28, 28) ------- (op: Conv)
[INFO: misc.py: 142]: nonlocal_conv3_1_out : (4, 512, 4, 28, 28) => nonlocal_conv3_1_bn : (4, 512, 4, 28, 28) ------- (op: AffineNd)
[INFO: misc.py: 142]: res3_1_branch2c_bn_trans_trans: (4, 512, 4, 28, 28) => nonlocal_conv3_1_sum : (4, 512, 4, 28, 28) ------- (op: Sum)
[INFO: misc.py: 142]: nonlocal_conv3_1_bn : (4, 512, 4, 28, 28) => nonlocal_conv3_1_sum : (4, 512, 4, 28, 28) ------|
[INFO: misc.py: 142]: nonlocal_conv3_1_sum : (4, 512, 4, 28, 28) => nonlocal_conv3_1_sum_trans : (1, 16, 512, 28, 28) ------- (op: Transpose)
[INFO: misc.py: 142]: nonlocal_conv3_1_sum_trans : (1, 16, 512, 28, 28) => nonlocal_conv3_1_sum_trans : (1, 16, 512, 28, 28) ------- (op: Reshape)
[INFO: misc.py: 142]: res3_1_branch2c_bn_trans_shape5d: (5,) => nonlocal_conv3_1_sum_trans : (1, 16, 512, 28, 28) ------|
[INFO: misc.py: 142]: nonlocal_conv3_1_sum_trans : (1, 16, 512, 28, 28) => nonlocal_conv3_1_sum_trans_trans : (1, 512, 16, 28, 28) ------- (op: Transpose)
[INFO: misc.py: 142]: nonlocal_conv3_1_sum_trans_trans: (1, 512, 16, 28, 28) => res3_2_branch2a : (1, 128, 16, 28, 28) ------- (op: Conv)
[INFO: misc.py: 142]: res3_2_branch2a : (1, 128, 16, 28, 28) => res3_2_branch2a_bn : (1, 128, 16, 28, 28) ------- (op: AffineNd)
[INFO: misc.py: 142]: res3_2_branch2a_bn : (1, 128, 16, 28, 28) => res3_2_branch2a_bn : (1, 128, 16, 28, 28) ------- (op: Relu)
[INFO: misc.py: 142]: res3_2_branch2a_bn : (1, 128, 16, 28, 28) => res3_2_branch2b : (1, 128, 16, 28, 28) ------- (op: Conv)
[INFO: misc.py: 142]: res3_2_branch2b : (1, 128, 16, 28, 28) => res3_2_branch2b_bn : (1, 128, 16, 28, 28) ------- (op: AffineNd)
[INFO: misc.py: 142]: res3_2_branch2b_bn : (1, 128, 16, 28, 28) => res3_2_branch2b_bn : (1, 128, 16, 28, 28) ------- (op: Relu)
[INFO: misc.py: 142]: res3_2_branch2b_bn : (1, 128, 16, 28, 28) => res3_2_branch2c : (1, 512, 16, 28, 28) ------- (op: Conv)
[INFO: misc.py: 142]: res3_2_branch2c : (1, 512, 16, 28, 28) => res3_2_branch2c_bn : (1, 512, 16, 28, 28) ------- (op: AffineNd)
[INFO: misc.py: 142]: res3_2_branch2c_bn : (1, 512, 16, 28, 28) => res3_2_branch2c_bn : (1, 512, 16, 28, 28) ------- (op: Sum)
[INFO: misc.py: 142]: nonlocal_conv3_1_sum_trans_trans: (1, 512, 16, 28, 28) => res3_2_branch2c_bn : (1, 512, 16, 28, 28) ------|
[INFO: misc.py: 142]: res3_2_branch2c_bn : (1, 512, 16, 28, 28) => res3_2_branch2c_bn : (1, 512, 16, 28, 28) ------- (op: Relu)
[INFO: misc.py: 142]: res3_2_branch2c_bn : (1, 512, 16, 28, 28) => res3_3_branch2a : (1, 128, 16, 28, 28) ------- (op: Conv)
[INFO: misc.py: 142]: res3_3_branch2a : (1, 128, 16, 28, 28) => res3_3_branch2a_bn : (1, 128, 16, 28, 28) ------- (op: AffineNd)
[INFO: misc.py: 142]: res3_3_branch2a_bn : (1, 128, 16, 28, 28) => res3_3_branch2a_bn : (1, 128, 16, 28, 28) ------- (op: Relu)
[INFO: misc.py: 142]: res3_3_branch2a_bn : (1, 128, 16, 28, 28) => res3_3_branch2b : (1, 128, 16, 28, 28) ------- (op: Conv)
[INFO: misc.py: 142]: res3_3_branch2b : (1, 128, 16, 28, 28) => res3_3_branch2b_bn : (1, 128, 16, 28, 28) ------- (op: AffineNd)
[INFO: misc.py: 142]: res3_3_branch2b_bn : (1, 128, 16, 28, 28) => res3_3_branch2b_bn : (1, 128, 16, 28, 28) ------- (op: Relu)
[INFO: misc.py: 142]: res3_3_branch2b_bn : (1, 128, 16, 28, 28) => res3_3_branch2c : (1, 512, 16, 28, 28) ------- (op: Conv)
[INFO: misc.py: 142]: res3_3_branch2c : (1, 512, 16, 28, 28) => res3_3_branch2c_bn : (1, 512, 16, 28, 28) ------- (op: AffineNd)
[INFO: misc.py: 142]: res3_3_branch2c_bn : (1, 512, 16, 28, 28) => res3_3_branch2c_bn : (1, 512, 16, 28, 28) ------- (op: Sum)
[INFO: misc.py: 142]: res3_2_branch2c_bn : (1, 512, 16, 28, 28) => res3_3_branch2c_bn : (1, 512, 16, 28, 28) ------|
[INFO: misc.py: 142]: res3_3_branch2c_bn : (1, 512, 16, 28, 28) => res3_3_branch2c_bn : (1, 512, 16, 28, 28) ------- (op: Relu)
[INFO: misc.py: 142]: res3_3_branch2c_bn : (1, 512, 16, 28, 28) => res3_3_branch2c_bn_trans : (4, 4, 512, 28, 28) ------- (op: Transpose)
[INFO: misc.py: 142]: res3_3_branch2c_bn_trans : (4, 4, 512, 28, 28) => res3_3_branch2c_bn_trans : (4, 4, 512, 28, 28) ------- (op: Reshape)
[INFO: misc.py: 142]: res3_3_branch2c_bn_trans : (4, 4, 512, 28, 28) => res3_3_branch2c_bn_trans_trans : (4, 512, 4, 28, 28) ------- (op: Transpose)
[INFO: misc.py: 142]: res3_3_branch2c_bn_trans_trans: (4, 512, 4, 28, 28) => nonlocal_conv3_3_theta : (4, 256, 3136) ------- (op: Conv)
[INFO: misc.py: 142]: res3_3_branch2c_bn_trans_trans: (4, 512, 4, 28, 28) => nonlocal_conv3_3_pool : (4, 512, 4, 14, 14) ------- (op: MaxPool)
[INFO: misc.py: 142]: nonlocal_conv3_3_pool : (4, 512, 4, 14, 14) => nonlocal_conv3_3_phi : (4, 256, 784) ------- (op: Conv)
[INFO: misc.py: 142]: nonlocal_conv3_3_pool : (4, 512, 4, 14, 14) => nonlocal_conv3_3_g : (4, 256, 784) ------- (op: Conv)
[INFO: misc.py: 142]: nonlocal_conv3_3_theta : (4, 256, 3136) => nonlocal_conv3_3_theta : (4, 256, 3136) ------- (op: Reshape)
[INFO: misc.py: 142]: nonlocal_conv3_3_phi : (4, 256, 784) => nonlocal_conv3_3_phi : (4, 256, 784) ------- (op: Reshape)
[INFO: misc.py: 142]: nonlocal_conv3_3_g : (4, 256, 784) => nonlocal_conv3_3_g : (4, 256, 784) ------- (op: Reshape)
[INFO: misc.py: 142]: nonlocal_conv3_3_theta : (4, 256, 3136) => nonlocal_conv3_3_affinity : (4, 3136, 784) ------- (op: BatchMatMul)
[INFO: misc.py: 142]: nonlocal_conv3_3_phi : (4, 256, 784) => nonlocal_conv3_3_affinity : (4, 3136, 784) ------|
[INFO: misc.py: 142]: nonlocal_conv3_3_affinity : (4, 3136, 784) => nonlocal_conv3_3_affinity : (4, 3136, 784) ------- (op: Scale)
[INFO: misc.py: 142]: nonlocal_conv3_3_affinity : (4, 3136, 784) => nonlocal_conv3_3_affinity_prob : (4, 3136, 784) ------- (op: Softmax)
[INFO: misc.py: 142]: nonlocal_conv3_3_g : (4, 256, 784) => nonlocal_conv3_3_y : (4, 256, 4, 28, 28) ------- (op: BatchMatMul)
[INFO: misc.py: 142]: nonlocal_conv3_3_affinity_prob: (4, 3136, 784) => nonlocal_conv3_3_y : (4, 256, 4, 28, 28) ------|
[INFO: misc.py: 142]: nonlocal_conv3_3_y : (4, 256, 4, 28, 28) => nonlocal_conv3_3_y : (4, 256, 4, 28, 28) ------- (op: Reshape)
[INFO: misc.py: 142]: nonlocal_conv3_3_theta_shape5d: (5,) => nonlocal_conv3_3_y : (4, 256, 4, 28, 28) ------|
[INFO: misc.py: 142]: nonlocal_conv3_3_y : (4, 256, 4, 28, 28) => nonlocal_conv3_3_out : (4, 512, 4, 28, 28) ------- (op: Conv)
[INFO: misc.py: 142]: nonlocal_conv3_3_out : (4, 512, 4, 28, 28) => nonlocal_conv3_3_bn : (4, 512, 4, 28, 28) ------- (op: AffineNd)
[INFO: misc.py: 142]: res3_3_branch2c_bn_trans_trans: (4, 512, 4, 28, 28) => nonlocal_conv3_3_sum : (4, 512, 4, 28, 28) ------- (op: Sum)
[INFO: misc.py: 142]: nonlocal_conv3_3_bn : (4, 512, 4, 28, 28) => nonlocal_conv3_3_sum : (4, 512, 4, 28, 28) ------|
[INFO: misc.py: 142]: nonlocal_conv3_3_sum : (4, 512, 4, 28, 28) => nonlocal_conv3_3_sum_trans : (1, 16, 512, 28, 28) ------- (op: Transpose)
[INFO: misc.py: 142]: nonlocal_conv3_3_sum_trans : (1, 16, 512, 28, 28) => nonlocal_conv3_3_sum_trans : (1, 16, 512, 28, 28) ------- (op: Reshape)
[INFO: misc.py: 142]: res3_3_branch2c_bn_trans_shape5d: (5,) => nonlocal_conv3_3_sum_trans : (1, 16, 512, 28, 28) ------|
[INFO: misc.py: 142]: nonlocal_conv3_3_sum_trans : (1, 16, 512, 28, 28) => nonlocal_conv3_3_sum_trans_trans : (1, 512, 16, 28, 28) ------- (op: Transpose)
[INFO: misc.py: 142]: nonlocal_conv3_3_sum_trans_trans: (1, 512, 16, 28, 28) => res4_0_branch2a : (1, 256, 16, 28, 28) ------- (op: Conv)
[INFO: misc.py: 142]: res4_0_branch2a : (1, 256, 16, 28, 28) => res4_0_branch2a_bn : (1, 256, 16, 28, 28) ------- (op: AffineNd)
[INFO: misc.py: 142]: res4_0_branch2a_bn : (1, 256, 16, 28, 28) => res4_0_branch2a_bn : (1, 256, 16, 28, 28) ------- (op: Relu)
[INFO: misc.py: 142]: res4_0_branch2a_bn : (1, 256, 16, 28, 28) => res4_0_branch2b : (1, 256, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: res4_0_branch2b : (1, 256, 16, 14, 14) => res4_0_branch2b_bn : (1, 256, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: res4_0_branch2b_bn : (1, 256, 16, 14, 14) => res4_0_branch2b_bn : (1, 256, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py: 142]: res4_0_branch2b_bn : (1, 256, 16, 14, 14) => res4_0_branch2c : (1, 1024, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: res4_0_branch2c : (1, 1024, 16, 14, 14) => res4_0_branch2c_bn : (1, 1024, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: nonlocal_conv3_3_sum_trans_trans: (1, 512, 16, 28, 28) => res4_0_branch1 : (1, 1024, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: res4_0_branch1 : (1, 1024, 16, 14, 14) => res4_0_branch1_bn : (1, 1024, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: res4_0_branch2c_bn : (1, 1024, 16, 14, 14) => res4_0_branch2c_bn : (1, 1024, 16, 14, 14) ------- (op: Sum)
[INFO: misc.py: 142]: res4_0_branch1_bn : (1, 1024, 16, 14, 14) => res4_0_branch2c_bn : (1, 1024, 16, 14, 14) ------|
[INFO: misc.py: 142]: res4_0_branch2c_bn : (1, 1024, 16, 14, 14) => res4_0_branch2c_bn : (1, 1024, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py: 142]: res4_0_branch2c_bn : (1, 1024, 16, 14, 14) => res4_1_branch2a : (1, 256, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: res4_1_branch2a : (1, 256, 16, 14, 14) => res4_1_branch2a_bn : (1, 256, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: res4_1_branch2a_bn : (1, 256, 16, 14, 14) => res4_1_branch2a_bn : (1, 256, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py: 142]: res4_1_branch2a_bn : (1, 256, 16, 14, 14) => res4_1_branch2b : (1, 256, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: res4_1_branch2b : (1, 256, 16, 14, 14) => res4_1_branch2b_bn : (1, 256, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: res4_1_branch2b_bn : (1, 256, 16, 14, 14) => res4_1_branch2b_bn : (1, 256, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py: 142]: res4_1_branch2b_bn : (1, 256, 16, 14, 14) => res4_1_branch2c : (1, 1024, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: res4_1_branch2c : (1, 1024, 16, 14, 14) => res4_1_branch2c_bn : (1, 1024, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: res4_1_branch2c_bn : (1, 1024, 16, 14, 14) => res4_1_branch2c_bn : (1, 1024, 16, 14, 14) ------- (op: Sum)
[INFO: misc.py: 142]: res4_0_branch2c_bn : (1, 1024, 16, 14, 14) => res4_1_branch2c_bn : (1, 1024, 16, 14, 14) ------|
[INFO: misc.py: 142]: res4_1_branch2c_bn : (1, 1024, 16, 14, 14) => res4_1_branch2c_bn : (1, 1024, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py: 142]: res4_1_branch2c_bn : (1, 1024, 16, 14, 14) => nonlocal_conv4_1_theta : (1, 512, 3136) ------- (op: Conv)
[INFO: misc.py: 142]: res4_1_branch2c_bn : (1, 1024, 16, 14, 14) => nonlocal_conv4_1_pool : (1, 1024, 16, 7, 7) ------- (op: MaxPool)
[INFO: misc.py: 142]: nonlocal_conv4_1_pool : (1, 1024, 16, 7, 7) => nonlocal_conv4_1_phi : (1, 512, 784) ------- (op: Conv)
[INFO: misc.py: 142]: nonlocal_conv4_1_pool : (1, 1024, 16, 7, 7) => nonlocal_conv4_1_g : (1, 512, 784) ------- (op: Conv)
[INFO: misc.py: 142]: nonlocal_conv4_1_theta : (1, 512, 3136) => nonlocal_conv4_1_theta : (1, 512, 3136) ------- (op: Reshape)
[INFO: misc.py: 142]: nonlocal_conv4_1_phi : (1, 512, 784) => nonlocal_conv4_1_phi : (1, 512, 784) ------- (op: Reshape)
[INFO: misc.py: 142]: nonlocal_conv4_1_g : (1, 512, 784) => nonlocal_conv4_1_g : (1, 512, 784) ------- (op: Reshape)
[INFO: misc.py: 142]: nonlocal_conv4_1_theta : (1, 512, 3136) => nonlocal_conv4_1_affinity : (1, 3136, 784) ------- (op: BatchMatMul)
[INFO: misc.py: 142]: nonlocal_conv4_1_phi : (1, 512, 784) => nonlocal_conv4_1_affinity : (1, 3136, 784) ------|
[INFO: misc.py: 142]: nonlocal_conv4_1_affinity : (1, 3136, 784) => nonlocal_conv4_1_affinity : (1, 3136, 784) ------- (op: Scale)
[INFO: misc.py: 142]: nonlocal_conv4_1_affinity : (1, 3136, 784) => nonlocal_conv4_1_affinity_prob : (1, 3136, 784) ------- (op: Softmax)
[INFO: misc.py: 142]: nonlocal_conv4_1_g : (1, 512, 784) => nonlocal_conv4_1_y : (1, 512, 16, 14, 14) ------- (op: BatchMatMul)
[INFO: misc.py: 142]: nonlocal_conv4_1_affinity_prob: (1, 3136, 784) => nonlocal_conv4_1_y : (1, 512, 16, 14, 14) ------|
[INFO: misc.py: 142]: nonlocal_conv4_1_y : (1, 512, 16, 14, 14) => nonlocal_conv4_1_y : (1, 512, 16, 14, 14) ------- (op: Reshape)
[INFO: misc.py: 142]: nonlocal_conv4_1_theta_shape5d: (5,) => nonlocal_conv4_1_y : (1, 512, 16, 14, 14) ------|
[INFO: misc.py: 142]: nonlocal_conv4_1_y : (1, 512, 16, 14, 14) => nonlocal_conv4_1_out : (1, 1024, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: nonlocal_conv4_1_out : (1, 1024, 16, 14, 14) => nonlocal_conv4_1_bn : (1, 1024, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: res4_1_branch2c_bn : (1, 1024, 16, 14, 14) => nonlocal_conv4_1_sum : (1, 1024, 16, 14, 14) ------- (op: Sum)
[INFO: misc.py: 142]: nonlocal_conv4_1_bn : (1, 1024, 16, 14, 14) => nonlocal_conv4_1_sum : (1, 1024, 16, 14, 14) ------|
[INFO: misc.py: 142]: nonlocal_conv4_1_sum : (1, 1024, 16, 14, 14) => res4_2_branch2a : (1, 256, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: res4_2_branch2a : (1, 256, 16, 14, 14) => res4_2_branch2a_bn : (1, 256, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: res4_2_branch2a_bn : (1, 256, 16, 14, 14) => res4_2_branch2a_bn : (1, 256, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py: 142]: res4_2_branch2a_bn : (1, 256, 16, 14, 14) => res4_2_branch2b : (1, 256, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: res4_2_branch2b : (1, 256, 16, 14, 14) => res4_2_branch2b_bn : (1, 256, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: res4_2_branch2b_bn : (1, 256, 16, 14, 14) => res4_2_branch2b_bn : (1, 256, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py: 142]: res4_2_branch2b_bn : (1, 256, 16, 14, 14) => res4_2_branch2c : (1, 1024, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: res4_2_branch2c : (1, 1024, 16, 14, 14) => res4_2_branch2c_bn : (1, 1024, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: res4_2_branch2c_bn : (1, 1024, 16, 14, 14) => res4_2_branch2c_bn : (1, 1024, 16, 14, 14) ------- (op: Sum)
[INFO: misc.py: 142]: nonlocal_conv4_1_sum : (1, 1024, 16, 14, 14) => res4_2_branch2c_bn : (1, 1024, 16, 14, 14) ------|
[INFO: misc.py: 142]: res4_2_branch2c_bn : (1, 1024, 16, 14, 14) => res4_2_branch2c_bn : (1, 1024, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py: 142]: res4_2_branch2c_bn : (1, 1024, 16, 14, 14) => res4_3_branch2a : (1, 256, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: res4_3_branch2a : (1, 256, 16, 14, 14) => res4_3_branch2a_bn : (1, 256, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: res4_3_branch2a_bn : (1, 256, 16, 14, 14) => res4_3_branch2a_bn : (1, 256, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py: 142]: res4_3_branch2a_bn : (1, 256, 16, 14, 14) => res4_3_branch2b : (1, 256, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: res4_3_branch2b : (1, 256, 16, 14, 14) => res4_3_branch2b_bn : (1, 256, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: res4_3_branch2b_bn : (1, 256, 16, 14, 14) => res4_3_branch2b_bn : (1, 256, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py: 142]: res4_3_branch2b_bn : (1, 256, 16, 14, 14) => res4_3_branch2c : (1, 1024, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: res4_3_branch2c : (1, 1024, 16, 14, 14) => res4_3_branch2c_bn : (1, 1024, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: res4_3_branch2c_bn : (1, 1024, 16, 14, 14) => res4_3_branch2c_bn : (1, 1024, 16, 14, 14) ------- (op: Sum)
[INFO: misc.py: 142]: res4_2_branch2c_bn : (1, 1024, 16, 14, 14) => res4_3_branch2c_bn : (1, 1024, 16, 14, 14) ------|
[INFO: misc.py: 142]: res4_3_branch2c_bn : (1, 1024, 16, 14, 14) => res4_3_branch2c_bn : (1, 1024, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py: 142]: res4_3_branch2c_bn : (1, 1024, 16, 14, 14) => nonlocal_conv4_3_theta : (1, 512, 3136) ------- (op: Conv)
[INFO: misc.py: 142]: res4_3_branch2c_bn : (1, 1024, 16, 14, 14) => nonlocal_conv4_3_pool : (1, 1024, 16, 7, 7) ------- (op: MaxPool)
[INFO: misc.py: 142]: nonlocal_conv4_3_pool : (1, 1024, 16, 7, 7) => nonlocal_conv4_3_phi : (1, 512, 784) ------- (op: Conv)
[INFO: misc.py: 142]: nonlocal_conv4_3_pool : (1, 1024, 16, 7, 7) => nonlocal_conv4_3_g : (1, 512, 784) ------- (op: Conv)
[INFO: misc.py: 142]: nonlocal_conv4_3_theta : (1, 512, 3136) => nonlocal_conv4_3_theta : (1, 512, 3136) ------- (op: Reshape)
[INFO: misc.py: 142]: nonlocal_conv4_3_phi : (1, 512, 784) => nonlocal_conv4_3_phi : (1, 512, 784) ------- (op: Reshape)
[INFO: misc.py: 142]: nonlocal_conv4_3_g : (1, 512, 784) => nonlocal_conv4_3_g : (1, 512, 784) ------- (op: Reshape)
[INFO: misc.py: 142]: nonlocal_conv4_3_theta : (1, 512, 3136) => nonlocal_conv4_3_affinity : (1, 3136, 784) ------- (op: BatchMatMul)
[INFO: misc.py: 142]: nonlocal_conv4_3_phi : (1, 512, 784) => nonlocal_conv4_3_affinity : (1, 3136, 784) ------|
[INFO: misc.py: 142]: nonlocal_conv4_3_affinity : (1, 3136, 784) => nonlocal_conv4_3_affinity : (1, 3136, 784) ------- (op: Scale)
[INFO: misc.py: 142]: nonlocal_conv4_3_affinity : (1, 3136, 784) => nonlocal_conv4_3_affinity_prob : (1, 3136, 784) ------- (op: Softmax)
[INFO: misc.py: 142]: nonlocal_conv4_3_g : (1, 512, 784) => nonlocal_conv4_3_y : (1, 512, 16, 14, 14) ------- (op: BatchMatMul)
[INFO: misc.py: 142]: nonlocal_conv4_3_affinity_prob: (1, 3136, 784) => nonlocal_conv4_3_y : (1, 512, 16, 14, 14) ------|
[INFO: misc.py: 142]: nonlocal_conv4_3_y : (1, 512, 16, 14, 14) => nonlocal_conv4_3_y : (1, 512, 16, 14, 14) ------- (op: Reshape)
[INFO: misc.py: 142]: nonlocal_conv4_3_theta_shape5d: (5,) => nonlocal_conv4_3_y : (1, 512, 16, 14, 14) ------|
[INFO: misc.py: 142]: nonlocal_conv4_3_y : (1, 512, 16, 14, 14) => nonlocal_conv4_3_out : (1, 1024, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: nonlocal_conv4_3_out : (1, 1024, 16, 14, 14) => nonlocal_conv4_3_bn : (1, 1024, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: res4_3_branch2c_bn : (1, 1024, 16, 14, 14) => nonlocal_conv4_3_sum : (1, 1024, 16, 14, 14) ------- (op: Sum)
[INFO: misc.py: 142]: nonlocal_conv4_3_bn : (1, 1024, 16, 14, 14) => nonlocal_conv4_3_sum : (1, 1024, 16, 14, 14) ------|
[INFO: misc.py: 142]: nonlocal_conv4_3_sum : (1, 1024, 16, 14, 14) => res4_4_branch2a : (1, 256, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: res4_4_branch2a : (1, 256, 16, 14, 14) => res4_4_branch2a_bn : (1, 256, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: res4_4_branch2a_bn : (1, 256, 16, 14, 14) => res4_4_branch2a_bn : (1, 256, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py: 142]: res4_4_branch2a_bn : (1, 256, 16, 14, 14) => res4_4_branch2b : (1, 256, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: res4_4_branch2b : (1, 256, 16, 14, 14) => res4_4_branch2b_bn : (1, 256, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: res4_4_branch2b_bn : (1, 256, 16, 14, 14) => res4_4_branch2b_bn : (1, 256, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py: 142]: res4_4_branch2b_bn : (1, 256, 16, 14, 14) => res4_4_branch2c : (1, 1024, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: res4_4_branch2c : (1, 1024, 16, 14, 14) => res4_4_branch2c_bn : (1, 1024, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: res4_4_branch2c_bn : (1, 1024, 16, 14, 14) => res4_4_branch2c_bn : (1, 1024, 16, 14, 14) ------- (op: Sum)
[INFO: misc.py: 142]: nonlocal_conv4_3_sum : (1, 1024, 16, 14, 14) => res4_4_branch2c_bn : (1, 1024, 16, 14, 14) ------|
[INFO: misc.py: 142]: res4_4_branch2c_bn : (1, 1024, 16, 14, 14) => res4_4_branch2c_bn : (1, 1024, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py: 142]: res4_4_branch2c_bn : (1, 1024, 16, 14, 14) => res4_5_branch2a : (1, 256, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: res4_5_branch2a : (1, 256, 16, 14, 14) => res4_5_branch2a_bn : (1, 256, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: res4_5_branch2a_bn : (1, 256, 16, 14, 14) => res4_5_branch2a_bn : (1, 256, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py: 142]: res4_5_branch2a_bn : (1, 256, 16, 14, 14) => res4_5_branch2b : (1, 256, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: res4_5_branch2b : (1, 256, 16, 14, 14) => res4_5_branch2b_bn : (1, 256, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: res4_5_branch2b_bn : (1, 256, 16, 14, 14) => res4_5_branch2b_bn : (1, 256, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py: 142]: res4_5_branch2b_bn : (1, 256, 16, 14, 14) => res4_5_branch2c : (1, 1024, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: res4_5_branch2c : (1, 1024, 16, 14, 14) => res4_5_branch2c_bn : (1, 1024, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: res4_5_branch2c_bn : (1, 1024, 16, 14, 14) => res4_5_branch2c_bn : (1, 1024, 16, 14, 14) ------- (op: Sum)
[INFO: misc.py: 142]: res4_4_branch2c_bn : (1, 1024, 16, 14, 14) => res4_5_branch2c_bn : (1, 1024, 16, 14, 14) ------|
[INFO: misc.py: 142]: res4_5_branch2c_bn : (1, 1024, 16, 14, 14) => res4_5_branch2c_bn : (1, 1024, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py: 142]: res4_5_branch2c_bn : (1, 1024, 16, 14, 14) => nonlocal_conv4_5_theta : (1, 512, 3136) ------- (op: Conv)
[INFO: misc.py: 142]: res4_5_branch2c_bn : (1, 1024, 16, 14, 14) => nonlocal_conv4_5_pool : (1, 1024, 16, 7, 7) ------- (op: MaxPool)
[INFO: misc.py: 142]: nonlocal_conv4_5_pool : (1, 1024, 16, 7, 7) => nonlocal_conv4_5_phi : (1, 512, 784) ------- (op: Conv)
[INFO: misc.py: 142]: nonlocal_conv4_5_pool : (1, 1024, 16, 7, 7) => nonlocal_conv4_5_g : (1, 512, 784) ------- (op: Conv)
[INFO: misc.py: 142]: nonlocal_conv4_5_theta : (1, 512, 3136) => nonlocal_conv4_5_theta : (1, 512, 3136) ------- (op: Reshape)
[INFO: misc.py: 142]: nonlocal_conv4_5_phi : (1, 512, 784) => nonlocal_conv4_5_phi : (1, 512, 784) ------- (op: Reshape)
[INFO: misc.py: 142]: nonlocal_conv4_5_g : (1, 512, 784) => nonlocal_conv4_5_g : (1, 512, 784) ------- (op: Reshape)
[INFO: misc.py: 142]: nonlocal_conv4_5_theta : (1, 512, 3136) => nonlocal_conv4_5_affinity : (1, 3136, 784) ------- (op: BatchMatMul)
[INFO: misc.py: 142]: nonlocal_conv4_5_phi : (1, 512, 784) => nonlocal_conv4_5_affinity : (1, 3136, 784) ------|
[INFO: misc.py: 142]: nonlocal_conv4_5_affinity : (1, 3136, 784) => nonlocal_conv4_5_affinity : (1, 3136, 784) ------- (op: Scale)
[INFO: misc.py: 142]: nonlocal_conv4_5_affinity : (1, 3136, 784) => nonlocal_conv4_5_affinity_prob : (1, 3136, 784) ------- (op: Softmax)
[INFO: misc.py: 142]: nonlocal_conv4_5_g : (1, 512, 784) => nonlocal_conv4_5_y : (1, 512, 16, 14, 14) ------- (op: BatchMatMul)
[INFO: misc.py: 142]: nonlocal_conv4_5_affinity_prob: (1, 3136, 784) => nonlocal_conv4_5_y : (1, 512, 16, 14, 14) ------|
[INFO: misc.py: 142]: nonlocal_conv4_5_y : (1, 512, 16, 14, 14) => nonlocal_conv4_5_y : (1, 512, 16, 14, 14) ------- (op: Reshape)
[INFO: misc.py: 142]: nonlocal_conv4_5_theta_shape5d: (5,) => nonlocal_conv4_5_y : (1, 512, 16, 14, 14) ------|
[INFO: misc.py: 142]: nonlocal_conv4_5_y : (1, 512, 16, 14, 14) => nonlocal_conv4_5_out : (1, 1024, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: nonlocal_conv4_5_out : (1, 1024, 16, 14, 14) => nonlocal_conv4_5_bn : (1, 1024, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: res4_5_branch2c_bn : (1, 1024, 16, 14, 14) => nonlocal_conv4_5_sum : (1, 1024, 16, 14, 14) ------- (op: Sum)
[INFO: misc.py: 142]: nonlocal_conv4_5_bn : (1, 1024, 16, 14, 14) => nonlocal_conv4_5_sum : (1, 1024, 16, 14, 14) ------|
[INFO: misc.py: 142]: nonlocal_conv4_5_sum : (1, 1024, 16, 14, 14) => res5_0_branch2a : (1, 512, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: res5_0_branch2a : (1, 512, 16, 14, 14) => res5_0_branch2a_bn : (1, 512, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: res5_0_branch2a_bn : (1, 512, 16, 14, 14) => res5_0_branch2a_bn : (1, 512, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py: 142]: res5_0_branch2a_bn : (1, 512, 16, 14, 14) => res5_0_branch2b : (1, 512, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: res5_0_branch2b : (1, 512, 16, 14, 14) => res5_0_branch2b_bn : (1, 512, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: res5_0_branch2b_bn : (1, 512, 16, 14, 14) => res5_0_branch2b_bn : (1, 512, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py: 142]: res5_0_branch2b_bn : (1, 512, 16, 14, 14) => res5_0_branch2c : (1, 2048, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: res5_0_branch2c : (1, 2048, 16, 14, 14) => res5_0_branch2c_bn : (1, 2048, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: nonlocal_conv4_5_sum : (1, 1024, 16, 14, 14) => res5_0_branch1 : (1, 2048, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: res5_0_branch1 : (1, 2048, 16, 14, 14) => res5_0_branch1_bn : (1, 2048, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: res5_0_branch2c_bn : (1, 2048, 16, 14, 14) => res5_0_branch2c_bn : (1, 2048, 16, 14, 14) ------- (op: Sum)
[INFO: misc.py: 142]: res5_0_branch1_bn : (1, 2048, 16, 14, 14) => res5_0_branch2c_bn : (1, 2048, 16, 14, 14) ------|
[INFO: misc.py: 142]: res5_0_branch2c_bn : (1, 2048, 16, 14, 14) => res5_0_branch2c_bn : (1, 2048, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py: 142]: res5_0_branch2c_bn : (1, 2048, 16, 14, 14) => res5_1_branch2a : (1, 512, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: res5_1_branch2a : (1, 512, 16, 14, 14) => res5_1_branch2a_bn : (1, 512, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: res5_1_branch2a_bn : (1, 512, 16, 14, 14) => res5_1_branch2a_bn : (1, 512, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py: 142]: res5_1_branch2a_bn : (1, 512, 16, 14, 14) => res5_1_branch2b : (1, 512, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: res5_1_branch2b : (1, 512, 16, 14, 14) => res5_1_branch2b_bn : (1, 512, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: res5_1_branch2b_bn : (1, 512, 16, 14, 14) => res5_1_branch2b_bn : (1, 512, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py: 142]: res5_1_branch2b_bn : (1, 512, 16, 14, 14) => res5_1_branch2c : (1, 2048, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: res5_1_branch2c : (1, 2048, 16, 14, 14) => res5_1_branch2c_bn : (1, 2048, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: res5_1_branch2c_bn : (1, 2048, 16, 14, 14) => res5_1_branch2c_bn : (1, 2048, 16, 14, 14) ------- (op: Sum)
[INFO: misc.py: 142]: res5_0_branch2c_bn : (1, 2048, 16, 14, 14) => res5_1_branch2c_bn : (1, 2048, 16, 14, 14) ------|
[INFO: misc.py: 142]: res5_1_branch2c_bn : (1, 2048, 16, 14, 14) => res5_1_branch2c_bn : (1, 2048, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py: 142]: res5_1_branch2c_bn : (1, 2048, 16, 14, 14) => res5_2_branch2a : (1, 512, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: res5_2_branch2a : (1, 512, 16, 14, 14) => res5_2_branch2a_bn : (1, 512, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: res5_2_branch2a_bn : (1, 512, 16, 14, 14) => res5_2_branch2a_bn : (1, 512, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py: 142]: res5_2_branch2a_bn : (1, 512, 16, 14, 14) => res5_2_branch2b : (1, 512, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: res5_2_branch2b : (1, 512, 16, 14, 14) => res5_2_branch2b_bn : (1, 512, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: res5_2_branch2b_bn : (1, 512, 16, 14, 14) => res5_2_branch2b_bn : (1, 512, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py: 142]: res5_2_branch2b_bn : (1, 512, 16, 14, 14) => res5_2_branch2c : (1, 2048, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py: 142]: res5_2_branch2c : (1, 2048, 16, 14, 14) => res5_2_branch2c_bn : (1, 2048, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py: 142]: res5_2_branch2c_bn : (1, 2048, 16, 14, 14) => res5_2_branch2c_bn : (1, 2048, 16, 14, 14) ------- (op: Sum)
[INFO: misc.py: 142]: res5_1_branch2c_bn : (1, 2048, 16, 14, 14) => res5_2_branch2c_bn : (1, 2048, 16, 14, 14) ------|
[INFO: misc.py: 142]: res5_2_branch2c_bn : (1, 2048, 16, 14, 14) => res5_2_branch2c_bn : (1, 2048, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py: 142]: res5_2_branch2c_bn : (1, 2048, 16, 14, 14) => res5_2_branch2c_bn_pooled : (1, 2048, 1, 1, 1) ------- (op: AveragePool)
[INFO: misc.py: 142]: res5_2_branch2c_bn_pooled : (1, 2048, 1, 1, 1) => pool5 : (1, 2048, 1, 1, 1) ------- (op: Concat)
[INFO: misc.py: 142]: pool5 : (1, 2048, 1, 1, 1) => pool5_dropout : (1, 2048, 1, 1, 1) ------- (op: Dropout)
[INFO: misc.py: 142]: pool5_dropout : (1, 2048, 1, 1, 1) => pred : (1, 157) ------- (op: FC)
[INFO: misc.py: 142]: pred : (1, 157) => prob : (1, 157) ------- (op: Sigmoid)
[INFO: misc.py: 142]: pred : (1, 157) => loss : () ------- (op: SigmoidCrossEntropyLoss)
[INFO: misc.py: 142]: labels_train : (1, 157) => loss : () ------|
[INFO: misc.py: 142]: pred : (1, 157) => __m14_shared : (1, 2048, 16, 14, 14) ------- (op: SigmoidCrossEntropyLossGradient)
[INFO: misc.py: 142]: labels_train : (1, 157) => __m14_shared : (1, 2048, 16, 14, 14) ------|
[INFO: misc.py: 142]: __m6_shared : (1, 3, 32, 224, 224) => __m14_shared : (1, 2048, 16, 14, 14) ------|
[INFO: misc.py: 147]: End of model: CNN_1
Mon Aug 2 10:38:38 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.32.03 Driver Version: 460.32.03 CUDA Version: 11.2 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Tesla V100-SXM2... On | 00000000:3D:00.0 Off | 0 |
| N/A 39C P0 58W / 300W | 13178MiB / 16160MiB | 6% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 1 Tesla V100-SXM2... On | 00000000:3E:00.0 Off | 0 |
| N/A 38C P0 59W / 300W | 13134MiB / 16160MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 2 Tesla V100-SXM2... On | 00000000:B1:00.0 Off | 0 |
| N/A 38C P0 58W / 300W | 13134MiB / 16160MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 3 Tesla V100-SXM2... On | 00000000:B2:00.0 Off | 0 |
| N/A 38C P0 59W / 300W | 13134MiB / 16160MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 41058 C python 13175MiB |
| 1 N/A N/A 41058 C python 13131MiB |
| 2 N/A N/A 41058 C python 13131MiB |
| 3 N/A N/A 41058 C python 13131MiB |
+-----------------------------------------------------------------------------+
[INFO: misc.py: 268]: layer gpu_0/conv1 (1) FLOPs: 18882.23 M PARAMs: 47.04 K
[INFO: misc.py: 268]: layer gpu_0/res2_0_branch2a (1) FLOPs: 1233.13 M PARAMs: 12.29 K
[INFO: misc.py: 268]: layer gpu_0/res2_0_branch2b (1) FLOPs: 3699.38 M PARAMs: 36.86 K
[INFO: misc.py: 268]: layer gpu_0/res2_0_branch2c (1) FLOPs: 1644.17 M PARAMs: 16.38 K
[INFO: misc.py: 268]: layer gpu_0/res2_0_branch1 (1) FLOPs: 1644.17 M PARAMs: 16.38 K
[INFO: misc.py: 268]: layer gpu_0/res2_1_branch2a (1) FLOPs: 4932.50 M PARAMs: 49.15 K
[INFO: misc.py: 268]: layer gpu_0/res2_1_branch2b (1) FLOPs: 3699.38 M PARAMs: 36.86 K
[INFO: misc.py: 268]: layer gpu_0/res2_1_branch2c (1) FLOPs: 1644.17 M PARAMs: 16.38 K
[INFO: misc.py: 268]: layer gpu_0/res2_2_branch2a (1) FLOPs: 4932.50 M PARAMs: 49.15 K
[INFO: misc.py: 268]: layer gpu_0/res2_2_branch2b (1) FLOPs: 3699.38 M PARAMs: 36.86 K
[INFO: misc.py: 268]: layer gpu_0/res2_2_branch2c (1) FLOPs: 1644.17 M PARAMs: 16.38 K
[INFO: misc.py: 268]: layer gpu_0/res3_0_branch2a (1) FLOPs: 4932.50 M PARAMs: 98.30 K
[INFO: misc.py: 268]: layer gpu_0/res3_0_branch2b (1) FLOPs: 1849.69 M PARAMs: 147.46 K
[INFO: misc.py: 268]: layer gpu_0/res3_0_branch2c (1) FLOPs: 822.08 M PARAMs: 65.54 K
[INFO: misc.py: 268]: layer gpu_0/res3_0_branch1 (1) FLOPs: 1644.17 M PARAMs: 131.07 K
[INFO: misc.py: 268]: layer gpu_0/res3_1_branch2a (1) FLOPs: 822.08 M PARAMs: 65.54 K
[INFO: misc.py: 268]: layer gpu_0/res3_1_branch2b (1) FLOPs: 1849.69 M PARAMs: 147.46 K
[INFO: misc.py: 268]: layer gpu_0/res3_1_branch2c (1) FLOPs: 822.08 M PARAMs: 65.54 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv3_1_theta (4) FLOPs: 1644.17 M PARAMs: 131.07 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv3_1_phi (4) FLOPs: 411.04 M PARAMs: 131.07 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv3_1_g (4) FLOPs: 411.04 M PARAMs: 131.07 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv3_1_affinity (4) FLOPs: 2517.63 M PARAMs: 0.00 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv3_1_y (4) FLOPs: 2517.63 M PARAMs: 0.00 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv3_1_out (4) FLOPs: 1644.17 M PARAMs: 131.07 K
[INFO: misc.py: 268]: layer gpu_0/res3_2_branch2a (1) FLOPs: 2466.25 M PARAMs: 196.61 K
[INFO: misc.py: 268]: layer gpu_0/res3_2_branch2b (1) FLOPs: 1849.69 M PARAMs: 147.46 K
[INFO: misc.py: 268]: layer gpu_0/res3_2_branch2c (1) FLOPs: 822.08 M PARAMs: 65.54 K
[INFO: misc.py: 268]: layer gpu_0/res3_3_branch2a (1) FLOPs: 822.08 M PARAMs: 65.54 K
[INFO: misc.py: 268]: layer gpu_0/res3_3_branch2b (1) FLOPs: 1849.69 M PARAMs: 147.46 K
[INFO: misc.py: 268]: layer gpu_0/res3_3_branch2c (1) FLOPs: 822.08 M PARAMs: 65.54 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv3_3_theta (4) FLOPs: 1644.17 M PARAMs: 131.07 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv3_3_phi (4) FLOPs: 411.04 M PARAMs: 131.07 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv3_3_g (4) FLOPs: 411.04 M PARAMs: 131.07 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv3_3_affinity (4) FLOPs: 2517.63 M PARAMs: 0.00 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv3_3_y (4) FLOPs: 2517.63 M PARAMs: 0.00 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv3_3_out (4) FLOPs: 1644.17 M PARAMs: 131.07 K
[INFO: misc.py: 268]: layer gpu_0/res4_0_branch2a (1) FLOPs: 4932.50 M PARAMs: 393.22 K
[INFO: misc.py: 268]: layer gpu_0/res4_0_branch2b (1) FLOPs: 1849.69 M PARAMs: 589.82 K
[INFO: misc.py: 268]: layer gpu_0/res4_0_branch2c (1) FLOPs: 822.08 M PARAMs: 262.14 K
[INFO: misc.py: 268]: layer gpu_0/res4_0_branch1 (1) FLOPs: 1644.17 M PARAMs: 524.29 K
[INFO: misc.py: 268]: layer gpu_0/res4_1_branch2a (1) FLOPs: 822.08 M PARAMs: 262.14 K
[INFO: misc.py: 268]: layer gpu_0/res4_1_branch2b (1) FLOPs: 1849.69 M PARAMs: 589.82 K
[INFO: misc.py: 268]: layer gpu_0/res4_1_branch2c (1) FLOPs: 822.08 M PARAMs: 262.14 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv4_1_theta (1) FLOPs: 1644.17 M PARAMs: 524.29 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv4_1_phi (1) FLOPs: 411.04 M PARAMs: 524.29 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv4_1_g (1) FLOPs: 411.04 M PARAMs: 524.29 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv4_1_affinity (1) FLOPs: 1258.82 M PARAMs: 0.00 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv4_1_y (1) FLOPs: 1258.82 M PARAMs: 0.00 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv4_1_out (1) FLOPs: 1644.17 M PARAMs: 524.29 K
[INFO: misc.py: 268]: layer gpu_0/res4_2_branch2a (1) FLOPs: 2466.25 M PARAMs: 786.43 K
[INFO: misc.py: 268]: layer gpu_0/res4_2_branch2b (1) FLOPs: 1849.69 M PARAMs: 589.82 K
[INFO: misc.py: 268]: layer gpu_0/res4_2_branch2c (1) FLOPs: 822.08 M PARAMs: 262.14 K
[INFO: misc.py: 268]: layer gpu_0/res4_3_branch2a (1) FLOPs: 822.08 M PARAMs: 262.14 K
[INFO: misc.py: 268]: layer gpu_0/res4_3_branch2b (1) FLOPs: 1849.69 M PARAMs: 589.82 K
[INFO: misc.py: 268]: layer gpu_0/res4_3_branch2c (1) FLOPs: 822.08 M PARAMs: 262.14 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv4_3_theta (1) FLOPs: 1644.17 M PARAMs: 524.29 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv4_3_phi (1) FLOPs: 411.04 M PARAMs: 524.29 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv4_3_g (1) FLOPs: 411.04 M PARAMs: 524.29 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv4_3_affinity (1) FLOPs: 1258.82 M PARAMs: 0.00 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv4_3_y (1) FLOPs: 1258.82 M PARAMs: 0.00 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv4_3_out (1) FLOPs: 1644.17 M PARAMs: 524.29 K
[INFO: misc.py: 268]: layer gpu_0/res4_4_branch2a (1) FLOPs: 2466.25 M PARAMs: 786.43 K
[INFO: misc.py: 268]: layer gpu_0/res4_4_branch2b (1) FLOPs: 1849.69 M PARAMs: 589.82 K
[INFO: misc.py: 268]: layer gpu_0/res4_4_branch2c (1) FLOPs: 822.08 M PARAMs: 262.14 K
[INFO: misc.py: 268]: layer gpu_0/res4_5_branch2a (1) FLOPs: 822.08 M PARAMs: 262.14 K
[INFO: misc.py: 268]: layer gpu_0/res4_5_branch2b (1) FLOPs: 1849.69 M PARAMs: 589.82 K
[INFO: misc.py: 268]: layer gpu_0/res4_5_branch2c (1) FLOPs: 822.08 M PARAMs: 262.14 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv4_5_theta (1) FLOPs: 1644.17 M PARAMs: 524.29 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv4_5_phi (1) FLOPs: 411.04 M PARAMs: 524.29 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv4_5_g (1) FLOPs: 411.04 M PARAMs: 524.29 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv4_5_affinity (1) FLOPs: 1258.82 M PARAMs: 0.00 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv4_5_y (1) FLOPs: 1258.82 M PARAMs: 0.00 K
[INFO: misc.py: 268]: layer gpu_0/nonlocal_conv4_5_out (1) FLOPs: 1644.17 M PARAMs: 524.29 K
[INFO: misc.py: 268]: layer gpu_0/res5_0_branch2a (1) FLOPs: 1644.17 M PARAMs: 524.29 K
[INFO: misc.py: 268]: layer gpu_0/res5_0_branch2b (1) FLOPs: 7398.75 M PARAMs: 2359.30 K
[INFO: misc.py: 268]: layer gpu_0/res5_0_branch2c (1) FLOPs: 3288.33 M PARAMs: 1048.58 K
[INFO: misc.py: 268]: layer gpu_0/res5_0_branch1 (1) FLOPs: 6576.67 M PARAMs: 2097.15 K
[INFO: misc.py: 268]: layer gpu_0/res5_1_branch2a (1) FLOPs: 9865.00 M PARAMs: 3145.73 K
[INFO: misc.py: 268]: layer gpu_0/res5_1_branch2b (1) FLOPs: 7398.75 M PARAMs: 2359.30 K
[INFO: misc.py: 268]: layer gpu_0/res5_1_branch2c (1) FLOPs: 3288.33 M PARAMs: 1048.58 K
[INFO: misc.py: 268]: layer gpu_0/res5_2_branch2a (1) FLOPs: 3288.33 M PARAMs: 1048.58 K
[INFO: misc.py: 268]: layer gpu_0/res5_2_branch2b (1) FLOPs: 7398.75 M PARAMs: 2359.30 K
[INFO: misc.py: 268]: layer gpu_0/res5_2_branch2c (1) FLOPs: 3288.33 M PARAMs: 1048.58 K
[INFO: misc.py: 268]: layer gpu_0/pred (1) FLOPs: 0.32 M PARAMs: 321.54 K
[INFO: misc.py: 169]: Total conv/fc/matMul FLOPs: 190.646642688(e9)
[INFO: misc.py: 170]: Total conv/fc params: 34.83232(e6)
Segmentation fault
(venv) [aac12391zj@g0034 video-long-term-feature-banks]$ Process PoolWorker-92:
Traceback (most recent call last):
File "/apps/centos7/python/2.7.18/lib/python2.7/multiprocessing/process.py", line 267, in _bootstrap
self.run()
File "/apps/centos7/python/2.7.18/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/apps/centos7/python/2.7.18/lib/python2.7/multiprocessing/pool.py", line 122, in worker
put((job, i, (False, wrapped)))
File "/apps/centos7/python/2.7.18/lib/python2.7/multiprocessing/queues.py", line 390, in put
return send(obj)
IOError: [Errno 32] Broken pipe
When running train_net.py as of Stage 1 at Charades two-stage training in GETTING_STARTED.md, I met this error.
I used python 2.7.18, cuda 10.0.130.1, cudnn 7.6.5. And, I installed PyTorch 1.4 based on this issue. I replaced both the GPU_NUM and BATCH_SIZE to 4 because of the limitation of our GPU setting.
I appreciate it if anyone provides any suggestion to solve this Segmentation fault and Broken pipe issue. Thank you so much.
The detailed log is shown in the following.