search

ros-industrial / easy_perception_deployment

A ROS2 package that accelerates the training and deployment of CV models for industries.
Apache License 2.0
53 stars 16 forks source link

Couldn't generate COCO-formatted dataset from labelled dataset #10

Closed Srikiranrao closed 3 years ago

Srikiranrao commented 3 years ago

Hi all,

While following the Create Custom Label List document section. I faced an issue. I couldn't generate the COCO-formatted datasets under the location data/datasets/custom_dataset when I clicked button labelled "Generate Dataset" from easy_perception_deployment GUI

Attached below are the dataset location Screenshot 3

Could you help me in solving this issue, Also I have attached the entire app log for your reference.

Thanks.

Screenshot from 2021-03-22 16-44-53

WARNING: [ ../l ] does not exist. Set Model to fasterrcnn Set Precision Level at: 2 Dataset directory not provided. Please choose Dataset. Set Model to fasterrcnn Dataset directory not provided. Please choose Dataset. Dataset directory not provided. Please choose Dataset. [INFO ] init:get_config:71 - Loading config file from: /home/delta/.labelmerc Output directory already exists: ../data/datasets/custom_dataset/train_dataset Output directory already exists: ../data/datasets/custom_dataset/val_dataset Dataset not properly restructured. Please restructure Dataset. Dataset not properly restructured. Please restructure Dataset. Dataset not properly restructured. Please restructure Dataset. [ERROR] - Please ensure there is /train_dataset and /val_dataset sub-directories in the selected dataset directory. Dataset not properly restructured. Please restructure Dataset. Dataset not properly restructured. Please restructure Dataset. [p2_trainer] - FOUND. Skipping installation. TrainFarm created under /home/delta/epd_ros2_ws/src/easy_perception_deployment/easy_perception_deployment/gui/trainer/P2TrainFarm 2021-03-22 16:30:48,668 maskrcnn_benchmark INFO: Using 1 GPUs 2021-03-22 16:30:48,668 maskrcnn_benchmark INFO: Namespace(config_file='configs/custom/fasterrcnn_training.yaml', distributed=False, local_rank=0, opts=[], skip_test=False) 2021-03-22 16:30:48,668 maskrcnn_benchmark INFO: Collecting env info (might take some time) 2021-03-22 16:30:50,513 maskrcnn_benchmark INFO: PyTorch version: 1.2.0 Is debug build: No CUDA used to build PyTorch: 10.0.130

OS: Ubuntu 20.04.2 LTS GCC version: (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0 CMake version: version 3.16.3

Python version: 3.6 Is CUDA available: Yes CUDA runtime version: 10.1.243 GPU models and configuration: GPU 0: Quadro P4000 Nvidia driver version: 460.39 cuDNN version: Could not collect

Versions of relevant libraries: [pip3] numpy==1.19.5 [pip3] torch==1.2.0 [pip3] torchvision==0.4.0a0 [conda] _pytorch_select 0.2 gpu_0
[conda] blas 1.0 mkl
[conda] mkl 2020.2 256
[conda] mkl-service 2.3.0 py36he8ac12f_0
[conda] mkl_fft 1.3.0 py36h54f3939_0
[conda] mkl_random 1.1.1 py36h0573a6f_0
[conda] pytorch 1.2.0 cuda100py36h938c94c_0
[conda] pytorch-nightly 1.0.0.dev20190328 py3.6_cuda10.0.130_cudnn7.4.2_0 pytorch [conda] torchvision 0.4.0 cuda100py36hecfc37a_0 Pillow (8.1.2) 2021-03-22 16:30:50,513 maskrcnn_benchmark INFO: Loaded configuration file configs/custom/fasterrcnn_training.yaml 2021-03-22 16:30:50,513 maskrcnn_benchmark INFO: MODEL: META_ARCHITECTURE: "GeneralizedRCNN" WEIGHT: "weights/e2e_faster_rcnn_R_50_FPN_1x_trimmed.pth" BACKBONE: CONV_BODY: "R-50-FPN" RESNETS: BACKBONE_OUT_CHANNELS: 256 RPN: USE_FPN: True ANCHOR_STRIDE: (4, 8, 16, 32, 64) PRE_NMS_TOP_N_TRAIN: 2000 PRE_NMS_TOP_N_TEST: 1000 POST_NMS_TOP_N_TEST: 1000 FPN_POST_NMS_TOP_N_TEST: 1000 ROI_HEADS: USE_FPN: True ROI_BOX_HEAD: POOLER_RESOLUTION: 7 POOLER_SCALES: (0.25, 0.125, 0.0625, 0.03125) POOLER_SAMPLING_RATIO: 2 FEATURE_EXTRACTOR: "FPN2MLPFeatureExtractor" PREDICTOR: "FPNPredictor" NUM_CLASSES: 4 #Change to your number of objects +2 DATASETS: TEST: ("coco_custom_val",) TRAIN: ("coco_custom_train",) DATALOADER: SIZE_DIVISIBILITY: 32 SOLVER: BASE_LR: 0.001 # learning rate after warmup WEIGHT_DECAY: 0.0001 STEPS: (1000,1500,2000,2500) # where you want your learning rate to decrease MAX_ITER: 3000 #number of iteration WARMUP_ITERS: 500 IMS_PER_BATCH: 1 TEST_PERIOD: 200 #run validation every steps CHECKPOINT_PERIOD: 200 #save model every steps TEST: IMS_PER_BATCH: 1 OUTPUT_DIR: "./weights/custom/" # saved weight output directory

2021-03-22 16:30:50,514 maskrcnn_benchmark INFO: Running with config: AMP_VERBOSE: False DATALOADER: ASPECT_RATIO_GROUPING: True NUM_WORKERS: 4 SIZE_DIVISIBILITY: 32 DATASETS: TEST: ('coco_custom_val',) TRAIN: ('coco_custom_train',) DTYPE: float32 INPUT: BRIGHTNESS: 0.0 CONTRAST: 0.0 HORIZONTAL_FLIP_PROB_TRAIN: 0.5 HUE: 0.0 MAX_SIZE_TEST: 1333 MAX_SIZE_TRAIN: 1333 MIN_SIZE_TEST: 800 MIN_SIZE_TRAIN: (800,) PIXEL_MEAN: [102.9801, 115.9465, 122.7717] PIXEL_STD: [1.0, 1.0, 1.0] SATURATION: 0.0 TO_BGR255: True VERTICAL_FLIP_PROB_TRAIN: 0.0 MODEL: BACKBONE: CONV_BODY: R-50-FPN FREEZE_CONV_BODY_AT: 2 CLS_AGNOSTIC_BBOX_REG: False DEVICE: cuda FBNET: ARCH: default ARCH_DEF: BN_TYPE: bn DET_HEAD_BLOCKS: [] DET_HEAD_LAST_SCALE: 1.0 DET_HEAD_STRIDE: 0 DW_CONV_SKIP_BN: True DW_CONV_SKIP_RELU: True KPTS_HEAD_BLOCKS: [] KPTS_HEAD_LAST_SCALE: 0.0 KPTS_HEAD_STRIDE: 0 MASK_HEAD_BLOCKS: [] MASK_HEAD_LAST_SCALE: 0.0 MASK_HEAD_STRIDE: 0 RPN_BN_TYPE: RPN_HEAD_BLOCKS: 0 SCALE_FACTOR: 1.0 WIDTH_DIVISOR: 1 FPN: USE_GN: False USE_RELU: False GROUP_NORM: DIM_PER_GP: -1 EPSILON: 1e-05 NUM_GROUPS: 32 KEYPOINT_ON: False MASK_ON: False META_ARCHITECTURE: GeneralizedRCNN RESNETS: BACKBONE_OUT_CHANNELS: 256 DEFORMABLE_GROUPS: 1 NUM_GROUPS: 1 RES2_OUT_CHANNELS: 256 RES5_DILATION: 1 STAGE_WITH_DCN: (False, False, False, False) STEM_FUNC: StemWithFixedBatchNorm STEM_OUT_CHANNELS: 64 STRIDE_IN_1X1: True TRANS_FUNC: BottleneckWithFixedBatchNorm WIDTH_PER_GROUP: 64 WITH_MODULATED_DCN: False RETINANET: ANCHOR_SIZES: (32, 64, 128, 256, 512) ANCHOR_STRIDES: (8, 16, 32, 64, 128) ASPECT_RATIOS: (0.5, 1.0, 2.0) BBOX_REG_BETA: 0.11 BBOX_REG_WEIGHT: 4.0 BG_IOU_THRESHOLD: 0.4 FG_IOU_THRESHOLD: 0.5 INFERENCE_TH: 0.05 LOSS_ALPHA: 0.25 LOSS_GAMMA: 2.0 NMS_TH: 0.4 NUM_CLASSES: 81 NUM_CONVS: 4 OCTAVE: 2.0 PRE_NMS_TOP_N: 1000 PRIOR_PROB: 0.01 SCALES_PER_OCTAVE: 3 STRADDLE_THRESH: 0 USE_C5: True RETINANET_ON: False ROI_BOX_HEAD: CONV_HEAD_DIM: 256 DILATION: 1 FEATURE_EXTRACTOR: FPN2MLPFeatureExtractor MLP_HEAD_DIM: 1024 NUM_CLASSES: 4 NUM_STACKED_CONVS: 4 POOLER_RESOLUTION: 7 POOLER_SAMPLING_RATIO: 2 POOLER_SCALES: (0.25, 0.125, 0.0625, 0.03125) PREDICTOR: FPNPredictor USE_GN: False ROI_HEADS: BATCH_SIZE_PER_IMAGE: 512 BBOX_REG_WEIGHTS: (10.0, 10.0, 5.0, 5.0) BG_IOU_THRESHOLD: 0.5 DETECTIONS_PER_IMG: 100 FG_IOU_THRESHOLD: 0.5 NMS: 0.5 POSITIVE_FRACTION: 0.25 SCORE_THRESH: 0.05 USE_FPN: True ROI_KEYPOINT_HEAD: CONV_LAYERS: (512, 512, 512, 512, 512, 512, 512, 512) FEATURE_EXTRACTOR: KeypointRCNNFeatureExtractor MLP_HEAD_DIM: 1024 NUM_CLASSES: 17 POOLER_RESOLUTION: 14 POOLER_SAMPLING_RATIO: 0 POOLER_SCALES: (0.0625,) PREDICTOR: KeypointRCNNPredictor RESOLUTION: 14 SHARE_BOX_FEATURE_EXTRACTOR: True ROI_MASK_HEAD: CONV_LAYERS: (256, 256, 256, 256) DILATION: 1 FEATURE_EXTRACTOR: ResNet50Conv5ROIFeatureExtractor MLP_HEAD_DIM: 1024 POOLER_RESOLUTION: 14 POOLER_SAMPLING_RATIO: 0 POOLER_SCALES: (0.0625,) POSTPROCESS_MASKS: False POSTPROCESS_MASKS_THRESHOLD: 0.5 PREDICTOR: MaskRCNNC4Predictor RESOLUTION: 14 SHARE_BOX_FEATURE_EXTRACTOR: True USE_GN: False RPN: ANCHOR_SIZES: (32, 64, 128, 256, 512) ANCHOR_STRIDE: (4, 8, 16, 32, 64) ASPECT_RATIOS: (0.5, 1.0, 2.0) BATCH_SIZE_PER_IMAGE: 256 BG_IOU_THRESHOLD: 0.3 FG_IOU_THRESHOLD: 0.7 FPN_POST_NMS_PER_BATCH: True FPN_POST_NMS_TOP_N_TEST: 1000 FPN_POST_NMS_TOP_N_TRAIN: 2000 MIN_SIZE: 0 NMS_THRESH: 0.7 POSITIVE_FRACTION: 0.5 POST_NMS_TOP_N_TEST: 1000 POST_NMS_TOP_N_TRAIN: 2000 PRE_NMS_TOP_N_TEST: 1000 PRE_NMS_TOP_N_TRAIN: 2000 RPN_HEAD: SingleConvRPNHead STRADDLE_THRESH: 0 USE_FPN: True RPN_ONLY: False WEIGHT: weights/e2e_faster_rcnn_R_50_FPN_1x_trimmed.pth OUTPUT_DIR: ./weights/custom/ PATHS_CATALOG: /home/delta/anaconda3/envs/p2_trainer/lib/python3.6/site-packages/maskrcnn_benchmark-0.1-py3.6-linux-x86_64.egg/maskrcnn_benchmark/config/paths_catalog.py SOLVER: BASE_LR: 0.001 BIAS_LR_FACTOR: 2 CHECKPOINT_PERIOD: 200 GAMMA: 0.1 IMS_PER_BATCH: 1 MAX_ITER: 3000 MOMENTUM: 0.9 STEPS: (1000, 1500, 2000, 2500) TEST_PERIOD: 200 WARMUP_FACTOR: 0.3333333333333333 WARMUP_ITERS: 500 WARMUP_METHOD: linear WEIGHT_DECAY: 0.0001 WEIGHT_DECAY_BIAS: 0 TEST: BBOX_AUG: ENABLED: False H_FLIP: False MAX_SIZE: 4000 SCALES: () SCALE_H_FLIP: False DETECTIONS_PER_IMG: 100 EXPECTED_RESULTS: [] EXPECTED_RESULTS_SIGMA_TOL: 4 IMS_PER_BATCH: 1 2021-03-22 16:30:50,514 maskrcnn_benchmark INFO: Saving config into: ./weights/custom/config.yml Selected optimization level O0: Pure FP32 training.

Defaults for this optimization level are: enabled : True opt_level : O0 cast_model_type : torch.float32 patch_torch_functions : False keep_batchnorm_fp32 : None master_weights : False loss_scale : 1.0 Processing user overrides (additional kwargs that are not None)... After processing overrides, optimization options are: enabled : True opt_level : O0 cast_model_type : torch.float32 patch_torch_functions : False keep_batchnorm_fp32 : None master_weights : False loss_scale : 1.0 Warning: multi_tensor_applier fused unscale kernel is unavailable, possibly because apex was installed without --cuda_ext --cpp_ext. Using Python fallback. Original ImportError was: ModuleNotFoundError("No module named 'amp_C'",) 2021-03-22 16:30:52,011 maskrcnn_benchmark.utils.checkpoint INFO: Loading checkpoint from weights/e2e_faster_rcnn_R_50_FPN_1x_trimmed.pth 2021-03-22 16:30:53,714 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.0.bn1.bias loaded from backbone.body.layer1.0.bn1.bias of shape (64,) 2021-03-22 16:30:53,714 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.0.bn1.running_mean loaded from backbone.body.layer1.0.bn1.running_mean of shape (64,) 2021-03-22 16:30:53,714 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.0.bn1.running_var loaded from backbone.body.layer1.0.bn1.running_var of shape (64,) 2021-03-22 16:30:53,714 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.0.bn1.weight loaded from backbone.body.layer1.0.bn1.weight of shape (64,) 2021-03-22 16:30:53,714 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.0.bn2.bias loaded from backbone.body.layer1.0.bn2.bias of shape (64,) 2021-03-22 16:30:53,714 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.0.bn2.running_mean loaded from backbone.body.layer1.0.bn2.running_mean of shape (64,) 2021-03-22 16:30:53,714 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.0.bn2.running_var loaded from backbone.body.layer1.0.bn2.running_var of shape (64,) 2021-03-22 16:30:53,714 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.0.bn2.weight loaded from backbone.body.layer1.0.bn2.weight of shape (64,) 2021-03-22 16:30:53,714 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.0.bn3.bias loaded from backbone.body.layer1.0.bn3.bias of shape (256,) 2021-03-22 16:30:53,714 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.0.bn3.running_mean loaded from backbone.body.layer1.0.bn3.running_mean of shape (256,) 2021-03-22 16:30:53,714 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.0.bn3.running_var loaded from backbone.body.layer1.0.bn3.running_var of shape (256,) 2021-03-22 16:30:53,714 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.0.bn3.weight loaded from backbone.body.layer1.0.bn3.weight of shape (256,) 2021-03-22 16:30:53,714 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.0.conv1.weight loaded from backbone.body.layer1.0.conv1.weight of shape (64, 64, 1, 1) 2021-03-22 16:30:53,714 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.0.conv2.weight loaded from backbone.body.layer1.0.conv2.weight of shape (64, 64, 3, 3) 2021-03-22 16:30:53,714 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.0.conv3.weight loaded from backbone.body.layer1.0.conv3.weight of shape (256, 64, 1, 1) 2021-03-22 16:30:53,714 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.0.downsample.0.weight loaded from backbone.body.layer1.0.downsample.0.weight of shape (256, 64, 1, 1) 2021-03-22 16:30:53,714 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.0.downsample.1.bias loaded from backbone.body.layer1.0.downsample.1.bias of shape (256,) 2021-03-22 16:30:53,715 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.0.downsample.1.running_mean loaded from backbone.body.layer1.0.downsample.1.running_mean of shape (256,) 2021-03-22 16:30:53,715 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.0.downsample.1.running_var loaded from backbone.body.layer1.0.downsample.1.running_var of shape (256,) 2021-03-22 16:30:53,715 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.0.downsample.1.weight loaded from backbone.body.layer1.0.downsample.1.weight of shape (256,) 2021-03-22 16:30:53,715 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.1.bn1.bias loaded from backbone.body.layer1.1.bn1.bias of shape (64,) 2021-03-22 16:30:53,715 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.1.bn1.running_mean loaded from backbone.body.layer1.1.bn1.running_mean of shape (64,) 2021-03-22 16:30:53,715 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.1.bn1.running_var loaded from backbone.body.layer1.1.bn1.running_var of shape (64,) 2021-03-22 16:30:53,715 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.1.bn1.weight loaded from backbone.body.layer1.1.bn1.weight of shape (64,) 2021-03-22 16:30:53,715 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.1.bn2.bias loaded from backbone.body.layer1.1.bn2.bias of shape (64,) 2021-03-22 16:30:53,715 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.1.bn2.running_mean loaded from backbone.body.layer1.1.bn2.running_mean of shape (64,) 2021-03-22 16:30:53,715 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.1.bn2.running_var loaded from backbone.body.layer1.1.bn2.running_var of shape (64,) 2021-03-22 16:30:53,715 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.1.bn2.weight loaded from backbone.body.layer1.1.bn2.weight of shape (64,) 2021-03-22 16:30:53,715 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.1.bn3.bias loaded from backbone.body.layer1.1.bn3.bias of shape (256,) 2021-03-22 16:30:53,715 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.1.bn3.running_mean loaded from backbone.body.layer1.1.bn3.running_mean of shape (256,) 2021-03-22 16:30:53,715 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.1.bn3.running_var loaded from backbone.body.layer1.1.bn3.running_var of shape (256,) 2021-03-22 16:30:53,715 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.1.bn3.weight loaded from backbone.body.layer1.1.bn3.weight of shape (256,) 2021-03-22 16:30:53,715 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.1.conv1.weight loaded from backbone.body.layer1.1.conv1.weight of shape (64, 256, 1, 1) 2021-03-22 16:30:53,715 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.1.conv2.weight loaded from backbone.body.layer1.1.conv2.weight of shape (64, 64, 3, 3) 2021-03-22 16:30:53,715 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.1.conv3.weight loaded from backbone.body.layer1.1.conv3.weight of shape (256, 64, 1, 1) 2021-03-22 16:30:53,715 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.2.bn1.bias loaded from backbone.body.layer1.2.bn1.bias of shape (64,) 2021-03-22 16:30:53,715 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.2.bn1.running_mean loaded from backbone.body.layer1.2.bn1.running_mean of shape (64,) 2021-03-22 16:30:53,715 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.2.bn1.running_var loaded from backbone.body.layer1.2.bn1.running_var of shape (64,) 2021-03-22 16:30:53,715 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.2.bn1.weight loaded from backbone.body.layer1.2.bn1.weight of shape (64,) 2021-03-22 16:30:53,715 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.2.bn2.bias loaded from backbone.body.layer1.2.bn2.bias of shape (64,) 2021-03-22 16:30:53,715 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.2.bn2.running_mean loaded from backbone.body.layer1.2.bn2.running_mean of shape (64,) 2021-03-22 16:30:53,715 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.2.bn2.running_var loaded from backbone.body.layer1.2.bn2.running_var of shape (64,) 2021-03-22 16:30:53,716 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.2.bn2.weight loaded from backbone.body.layer1.2.bn2.weight of shape (64,) 2021-03-22 16:30:53,716 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.2.bn3.bias loaded from backbone.body.layer1.2.bn3.bias of shape (256,) 2021-03-22 16:30:53,716 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.2.bn3.running_mean loaded from backbone.body.layer1.2.bn3.running_mean of shape (256,) 2021-03-22 16:30:53,716 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.2.bn3.running_var loaded from backbone.body.layer1.2.bn3.running_var of shape (256,) 2021-03-22 16:30:53,716 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.2.bn3.weight loaded from backbone.body.layer1.2.bn3.weight of shape (256,) 2021-03-22 16:30:53,716 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.2.conv1.weight loaded from backbone.body.layer1.2.conv1.weight of shape (64, 256, 1, 1) 2021-03-22 16:30:53,716 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.2.conv2.weight loaded from backbone.body.layer1.2.conv2.weight of shape (64, 64, 3, 3) 2021-03-22 16:30:53,716 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer1.2.conv3.weight loaded from backbone.body.layer1.2.conv3.weight of shape (256, 64, 1, 1) 2021-03-22 16:30:53,716 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.0.bn1.bias loaded from backbone.body.layer2.0.bn1.bias of shape (128,) 2021-03-22 16:30:53,716 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.0.bn1.running_mean loaded from backbone.body.layer2.0.bn1.running_mean of shape (128,) 2021-03-22 16:30:53,716 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.0.bn1.running_var loaded from backbone.body.layer2.0.bn1.running_var of shape (128,) 2021-03-22 16:30:53,716 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.0.bn1.weight loaded from backbone.body.layer2.0.bn1.weight of shape (128,) 2021-03-22 16:30:53,716 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.0.bn2.bias loaded from backbone.body.layer2.0.bn2.bias of shape (128,) 2021-03-22 16:30:53,716 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.0.bn2.running_mean loaded from backbone.body.layer2.0.bn2.running_mean of shape (128,) 2021-03-22 16:30:53,716 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.0.bn2.running_var loaded from backbone.body.layer2.0.bn2.running_var of shape (128,) 2021-03-22 16:30:53,716 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.0.bn2.weight loaded from backbone.body.layer2.0.bn2.weight of shape (128,) 2021-03-22 16:30:53,716 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.0.bn3.bias loaded from backbone.body.layer2.0.bn3.bias of shape (512,) 2021-03-22 16:30:53,716 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.0.bn3.running_mean loaded from backbone.body.layer2.0.bn3.running_mean of shape (512,) 2021-03-22 16:30:53,716 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.0.bn3.running_var loaded from backbone.body.layer2.0.bn3.running_var of shape (512,) 2021-03-22 16:30:53,716 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.0.bn3.weight loaded from backbone.body.layer2.0.bn3.weight of shape (512,) 2021-03-22 16:30:53,716 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.0.conv1.weight loaded from backbone.body.layer2.0.conv1.weight of shape (128, 256, 1, 1) 2021-03-22 16:30:53,716 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.0.conv2.weight loaded from backbone.body.layer2.0.conv2.weight of shape (128, 128, 3, 3) 2021-03-22 16:30:53,716 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.0.conv3.weight loaded from backbone.body.layer2.0.conv3.weight of shape (512, 128, 1, 1) 2021-03-22 16:30:53,716 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.0.downsample.0.weight loaded from backbone.body.layer2.0.downsample.0.weight of shape (512, 256, 1, 1) 2021-03-22 16:30:53,717 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.0.downsample.1.bias loaded from backbone.body.layer2.0.downsample.1.bias of shape (512,) 2021-03-22 16:30:53,717 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.0.downsample.1.running_mean loaded from backbone.body.layer2.0.downsample.1.running_mean of shape (512,) 2021-03-22 16:30:53,717 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.0.downsample.1.running_var loaded from backbone.body.layer2.0.downsample.1.running_var of shape (512,) 2021-03-22 16:30:53,717 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.0.downsample.1.weight loaded from backbone.body.layer2.0.downsample.1.weight of shape (512,) 2021-03-22 16:30:53,717 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.1.bn1.bias loaded from backbone.body.layer2.1.bn1.bias of shape (128,) 2021-03-22 16:30:53,717 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.1.bn1.running_mean loaded from backbone.body.layer2.1.bn1.running_mean of shape (128,) 2021-03-22 16:30:53,717 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.1.bn1.running_var loaded from backbone.body.layer2.1.bn1.running_var of shape (128,) 2021-03-22 16:30:53,717 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.1.bn1.weight loaded from backbone.body.layer2.1.bn1.weight of shape (128,) 2021-03-22 16:30:53,717 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.1.bn2.bias loaded from backbone.body.layer2.1.bn2.bias of shape (128,) 2021-03-22 16:30:53,717 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.1.bn2.running_mean loaded from backbone.body.layer2.1.bn2.running_mean of shape (128,) 2021-03-22 16:30:53,717 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.1.bn2.running_var loaded from backbone.body.layer2.1.bn2.running_var of shape (128,) 2021-03-22 16:30:53,717 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.1.bn2.weight loaded from backbone.body.layer2.1.bn2.weight of shape (128,) 2021-03-22 16:30:53,717 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.1.bn3.bias loaded from backbone.body.layer2.1.bn3.bias of shape (512,) 2021-03-22 16:30:53,717 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.1.bn3.running_mean loaded from backbone.body.layer2.1.bn3.running_mean of shape (512,) 2021-03-22 16:30:53,717 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.1.bn3.running_var loaded from backbone.body.layer2.1.bn3.running_var of shape (512,) 2021-03-22 16:30:53,717 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.1.bn3.weight loaded from backbone.body.layer2.1.bn3.weight of shape (512,) 2021-03-22 16:30:53,717 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.1.conv1.weight loaded from backbone.body.layer2.1.conv1.weight of shape (128, 512, 1, 1) 2021-03-22 16:30:53,717 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.1.conv2.weight loaded from backbone.body.layer2.1.conv2.weight of shape (128, 128, 3, 3) 2021-03-22 16:30:53,717 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.1.conv3.weight loaded from backbone.body.layer2.1.conv3.weight of shape (512, 128, 1, 1) 2021-03-22 16:30:53,717 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.2.bn1.bias loaded from backbone.body.layer2.2.bn1.bias of shape (128,) 2021-03-22 16:30:53,717 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.2.bn1.running_mean loaded from backbone.body.layer2.2.bn1.running_mean of shape (128,) 2021-03-22 16:30:53,717 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.2.bn1.running_var loaded from backbone.body.layer2.2.bn1.running_var of shape (128,) 2021-03-22 16:30:53,717 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.2.bn1.weight loaded from backbone.body.layer2.2.bn1.weight of shape (128,) 2021-03-22 16:30:53,717 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.2.bn2.bias loaded from backbone.body.layer2.2.bn2.bias of shape (128,) 2021-03-22 16:30:53,717 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.2.bn2.running_mean loaded from backbone.body.layer2.2.bn2.running_mean of shape (128,) 2021-03-22 16:30:53,718 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.2.bn2.running_var loaded from backbone.body.layer2.2.bn2.running_var of shape (128,) 2021-03-22 16:30:53,718 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.2.bn2.weight loaded from backbone.body.layer2.2.bn2.weight of shape (128,) 2021-03-22 16:30:53,718 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.2.bn3.bias loaded from backbone.body.layer2.2.bn3.bias of shape (512,) 2021-03-22 16:30:53,718 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.2.bn3.running_mean loaded from backbone.body.layer2.2.bn3.running_mean of shape (512,) 2021-03-22 16:30:53,718 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.2.bn3.running_var loaded from backbone.body.layer2.2.bn3.running_var of shape (512,) 2021-03-22 16:30:53,718 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.2.bn3.weight loaded from backbone.body.layer2.2.bn3.weight of shape (512,) 2021-03-22 16:30:53,718 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.2.conv1.weight loaded from backbone.body.layer2.2.conv1.weight of shape (128, 512, 1, 1) 2021-03-22 16:30:53,718 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.2.conv2.weight loaded from backbone.body.layer2.2.conv2.weight of shape (128, 128, 3, 3) 2021-03-22 16:30:53,718 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.2.conv3.weight loaded from backbone.body.layer2.2.conv3.weight of shape (512, 128, 1, 1) 2021-03-22 16:30:53,718 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.3.bn1.bias loaded from backbone.body.layer2.3.bn1.bias of shape (128,) 2021-03-22 16:30:53,718 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.3.bn1.running_mean loaded from backbone.body.layer2.3.bn1.running_mean of shape (128,) 2021-03-22 16:30:53,718 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.3.bn1.running_var loaded from backbone.body.layer2.3.bn1.running_var of shape (128,) 2021-03-22 16:30:53,718 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.3.bn1.weight loaded from backbone.body.layer2.3.bn1.weight of shape (128,) 2021-03-22 16:30:53,718 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.3.bn2.bias loaded from backbone.body.layer2.3.bn2.bias of shape (128,) 2021-03-22 16:30:53,718 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.3.bn2.running_mean loaded from backbone.body.layer2.3.bn2.running_mean of shape (128,) 2021-03-22 16:30:53,718 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.3.bn2.running_var loaded from backbone.body.layer2.3.bn2.running_var of shape (128,) 2021-03-22 16:30:53,718 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.3.bn2.weight loaded from backbone.body.layer2.3.bn2.weight of shape (128,) 2021-03-22 16:30:53,718 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.3.bn3.bias loaded from backbone.body.layer2.3.bn3.bias of shape (512,) 2021-03-22 16:30:53,718 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.3.bn3.running_mean loaded from backbone.body.layer2.3.bn3.running_mean of shape (512,) 2021-03-22 16:30:53,718 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.3.bn3.running_var loaded from backbone.body.layer2.3.bn3.running_var of shape (512,) 2021-03-22 16:30:53,718 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.3.bn3.weight loaded from backbone.body.layer2.3.bn3.weight of shape (512,) 2021-03-22 16:30:53,718 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.3.conv1.weight loaded from backbone.body.layer2.3.conv1.weight of shape (128, 512, 1, 1) 2021-03-22 16:30:53,718 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.3.conv2.weight loaded from backbone.body.layer2.3.conv2.weight of shape (128, 128, 3, 3) 2021-03-22 16:30:53,718 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer2.3.conv3.weight loaded from backbone.body.layer2.3.conv3.weight of shape (512, 128, 1, 1) 2021-03-22 16:30:53,718 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.0.bn1.bias loaded from backbone.body.layer3.0.bn1.bias of shape (256,) 2021-03-22 16:30:53,718 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.0.bn1.running_mean loaded from backbone.body.layer3.0.bn1.running_mean of shape (256,) 2021-03-22 16:30:53,718 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.0.bn1.running_var loaded from backbone.body.layer3.0.bn1.running_var of shape (256,) 2021-03-22 16:30:53,719 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.0.bn1.weight loaded from backbone.body.layer3.0.bn1.weight of shape (256,) 2021-03-22 16:30:53,719 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.0.bn2.bias loaded from backbone.body.layer3.0.bn2.bias of shape (256,) 2021-03-22 16:30:53,719 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.0.bn2.running_mean loaded from backbone.body.layer3.0.bn2.running_mean of shape (256,) 2021-03-22 16:30:53,719 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.0.bn2.running_var loaded from backbone.body.layer3.0.bn2.running_var of shape (256,) 2021-03-22 16:30:53,719 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.0.bn2.weight loaded from backbone.body.layer3.0.bn2.weight of shape (256,) 2021-03-22 16:30:53,719 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.0.bn3.bias loaded from backbone.body.layer3.0.bn3.bias of shape (1024,) 2021-03-22 16:30:53,719 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.0.bn3.running_mean loaded from backbone.body.layer3.0.bn3.running_mean of shape (1024,) 2021-03-22 16:30:53,719 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.0.bn3.running_var loaded from backbone.body.layer3.0.bn3.running_var of shape (1024,) 2021-03-22 16:30:53,719 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.0.bn3.weight loaded from backbone.body.layer3.0.bn3.weight of shape (1024,) 2021-03-22 16:30:53,719 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.0.conv1.weight loaded from backbone.body.layer3.0.conv1.weight of shape (256, 512, 1, 1) 2021-03-22 16:30:53,719 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.0.conv2.weight loaded from backbone.body.layer3.0.conv2.weight of shape (256, 256, 3, 3) 2021-03-22 16:30:53,719 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.0.conv3.weight loaded from backbone.body.layer3.0.conv3.weight of shape (1024, 256, 1, 1) 2021-03-22 16:30:53,719 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.0.downsample.0.weight loaded from backbone.body.layer3.0.downsample.0.weight of shape (1024, 512, 1, 1) 2021-03-22 16:30:53,719 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.0.downsample.1.bias loaded from backbone.body.layer3.0.downsample.1.bias of shape (1024,) 2021-03-22 16:30:53,719 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.0.downsample.1.running_mean loaded from backbone.body.layer3.0.downsample.1.running_mean of shape (1024,) 2021-03-22 16:30:53,719 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.0.downsample.1.running_var loaded from backbone.body.layer3.0.downsample.1.running_var of shape (1024,) 2021-03-22 16:30:53,719 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.0.downsample.1.weight loaded from backbone.body.layer3.0.downsample.1.weight of shape (1024,) 2021-03-22 16:30:53,719 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.1.bn1.bias loaded from backbone.body.layer3.1.bn1.bias of shape (256,) 2021-03-22 16:30:53,719 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.1.bn1.running_mean loaded from backbone.body.layer3.1.bn1.running_mean of shape (256,) 2021-03-22 16:30:53,719 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.1.bn1.running_var loaded from backbone.body.layer3.1.bn1.running_var of shape (256,) 2021-03-22 16:30:53,719 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.1.bn1.weight loaded from backbone.body.layer3.1.bn1.weight of shape (256,) 2021-03-22 16:30:53,719 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.1.bn2.bias loaded from backbone.body.layer3.1.bn2.bias of shape (256,) 2021-03-22 16:30:53,719 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.1.bn2.running_mean loaded from backbone.body.layer3.1.bn2.running_mean of shape (256,) 2021-03-22 16:30:53,719 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.1.bn2.running_var loaded from backbone.body.layer3.1.bn2.running_var of shape (256,) 2021-03-22 16:30:53,719 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.1.bn2.weight loaded from backbone.body.layer3.1.bn2.weight of shape (256,) 2021-03-22 16:30:53,719 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.1.bn3.bias loaded from backbone.body.layer3.1.bn3.bias of shape (1024,) 2021-03-22 16:30:53,719 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.1.bn3.running_mean loaded from backbone.body.layer3.1.bn3.running_mean of shape (1024,) 2021-03-22 16:30:53,720 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.1.bn3.running_var loaded from backbone.body.layer3.1.bn3.running_var of shape (1024,) 2021-03-22 16:30:53,720 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.1.bn3.weight loaded from backbone.body.layer3.1.bn3.weight of shape (1024,) 2021-03-22 16:30:53,720 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.1.conv1.weight loaded from backbone.body.layer3.1.conv1.weight of shape (256, 1024, 1, 1) 2021-03-22 16:30:53,720 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.1.conv2.weight loaded from backbone.body.layer3.1.conv2.weight of shape (256, 256, 3, 3) 2021-03-22 16:30:53,720 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.1.conv3.weight loaded from backbone.body.layer3.1.conv3.weight of shape (1024, 256, 1, 1) 2021-03-22 16:30:53,720 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.2.bn1.bias loaded from backbone.body.layer3.2.bn1.bias of shape (256,) 2021-03-22 16:30:53,720 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.2.bn1.running_mean loaded from backbone.body.layer3.2.bn1.running_mean of shape (256,) 2021-03-22 16:30:53,720 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.2.bn1.running_var loaded from backbone.body.layer3.2.bn1.running_var of shape (256,) 2021-03-22 16:30:53,720 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.2.bn1.weight loaded from backbone.body.layer3.2.bn1.weight of shape (256,) 2021-03-22 16:30:53,720 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.2.bn2.bias loaded from backbone.body.layer3.2.bn2.bias of shape (256,) 2021-03-22 16:30:53,720 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.2.bn2.running_mean loaded from backbone.body.layer3.2.bn2.running_mean of shape (256,) 2021-03-22 16:30:53,720 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.2.bn2.running_var loaded from backbone.body.layer3.2.bn2.running_var of shape (256,) 2021-03-22 16:30:53,720 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.2.bn2.weight loaded from backbone.body.layer3.2.bn2.weight of shape (256,) 2021-03-22 16:30:53,720 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.2.bn3.bias loaded from backbone.body.layer3.2.bn3.bias of shape (1024,) 2021-03-22 16:30:53,720 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.2.bn3.running_mean loaded from backbone.body.layer3.2.bn3.running_mean of shape (1024,) 2021-03-22 16:30:53,720 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.2.bn3.running_var loaded from backbone.body.layer3.2.bn3.running_var of shape (1024,) 2021-03-22 16:30:53,720 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.2.bn3.weight loaded from backbone.body.layer3.2.bn3.weight of shape (1024,) 2021-03-22 16:30:53,720 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.2.conv1.weight loaded from backbone.body.layer3.2.conv1.weight of shape (256, 1024, 1, 1) 2021-03-22 16:30:53,720 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.2.conv2.weight loaded from backbone.body.layer3.2.conv2.weight of shape (256, 256, 3, 3) 2021-03-22 16:30:53,720 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.2.conv3.weight loaded from backbone.body.layer3.2.conv3.weight of shape (1024, 256, 1, 1) 2021-03-22 16:30:53,720 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.3.bn1.bias loaded from backbone.body.layer3.3.bn1.bias of shape (256,) 2021-03-22 16:30:53,720 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.3.bn1.running_mean loaded from backbone.body.layer3.3.bn1.running_mean of shape (256,) 2021-03-22 16:30:53,720 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.3.bn1.running_var loaded from backbone.body.layer3.3.bn1.running_var of shape (256,) 2021-03-22 16:30:53,720 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.3.bn1.weight loaded from backbone.body.layer3.3.bn1.weight of shape (256,) 2021-03-22 16:30:53,720 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.3.bn2.bias loaded from backbone.body.layer3.3.bn2.bias of shape (256,) 2021-03-22 16:30:53,720 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.3.bn2.running_mean loaded from backbone.body.layer3.3.bn2.running_mean of shape (256,) 2021-03-22 16:30:53,720 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.3.bn2.running_var loaded from backbone.body.layer3.3.bn2.running_var of shape (256,) 2021-03-22 16:30:53,721 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.3.bn2.weight loaded from backbone.body.layer3.3.bn2.weight of shape (256,) 2021-03-22 16:30:53,721 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.3.bn3.bias loaded from backbone.body.layer3.3.bn3.bias of shape (1024,) 2021-03-22 16:30:53,721 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.3.bn3.running_mean loaded from backbone.body.layer3.3.bn3.running_mean of shape (1024,) 2021-03-22 16:30:53,721 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.3.bn3.running_var loaded from backbone.body.layer3.3.bn3.running_var of shape (1024,) 2021-03-22 16:30:53,721 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.3.bn3.weight loaded from backbone.body.layer3.3.bn3.weight of shape (1024,) 2021-03-22 16:30:53,721 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.3.conv1.weight loaded from backbone.body.layer3.3.conv1.weight of shape (256, 1024, 1, 1) 2021-03-22 16:30:53,721 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.3.conv2.weight loaded from backbone.body.layer3.3.conv2.weight of shape (256, 256, 3, 3) 2021-03-22 16:30:53,721 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.3.conv3.weight loaded from backbone.body.layer3.3.conv3.weight of shape (1024, 256, 1, 1) 2021-03-22 16:30:53,721 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.4.bn1.bias loaded from backbone.body.layer3.4.bn1.bias of shape (256,) 2021-03-22 16:30:53,721 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.4.bn1.running_mean loaded from backbone.body.layer3.4.bn1.running_mean of shape (256,) 2021-03-22 16:30:53,721 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.4.bn1.running_var loaded from backbone.body.layer3.4.bn1.running_var of shape (256,) 2021-03-22 16:30:53,721 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.4.bn1.weight loaded from backbone.body.layer3.4.bn1.weight of shape (256,) 2021-03-22 16:30:53,721 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.4.bn2.bias loaded from backbone.body.layer3.4.bn2.bias of shape (256,) 2021-03-22 16:30:53,721 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.4.bn2.running_mean loaded from backbone.body.layer3.4.bn2.running_mean of shape (256,) 2021-03-22 16:30:53,721 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.4.bn2.running_var loaded from backbone.body.layer3.4.bn2.running_var of shape (256,) 2021-03-22 16:30:53,721 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.4.bn2.weight loaded from backbone.body.layer3.4.bn2.weight of shape (256,) 2021-03-22 16:30:53,721 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.4.bn3.bias loaded from backbone.body.layer3.4.bn3.bias of shape (1024,) 2021-03-22 16:30:53,721 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.4.bn3.running_mean loaded from backbone.body.layer3.4.bn3.running_mean of shape (1024,) 2021-03-22 16:30:53,721 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.4.bn3.running_var loaded from backbone.body.layer3.4.bn3.running_var of shape (1024,) 2021-03-22 16:30:53,721 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.4.bn3.weight loaded from backbone.body.layer3.4.bn3.weight of shape (1024,) 2021-03-22 16:30:53,721 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.4.conv1.weight loaded from backbone.body.layer3.4.conv1.weight of shape (256, 1024, 1, 1) 2021-03-22 16:30:53,721 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.4.conv2.weight loaded from backbone.body.layer3.4.conv2.weight of shape (256, 256, 3, 3) 2021-03-22 16:30:53,721 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.4.conv3.weight loaded from backbone.body.layer3.4.conv3.weight of shape (1024, 256, 1, 1) 2021-03-22 16:30:53,721 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.5.bn1.bias loaded from backbone.body.layer3.5.bn1.bias of shape (256,) 2021-03-22 16:30:53,721 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.5.bn1.running_mean loaded from backbone.body.layer3.5.bn1.running_mean of shape (256,) 2021-03-22 16:30:53,721 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.5.bn1.running_var loaded from backbone.body.layer3.5.bn1.running_var of shape (256,) 2021-03-22 16:30:53,721 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.5.bn1.weight loaded from backbone.body.layer3.5.bn1.weight of shape (256,) 2021-03-22 16:30:53,721 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.5.bn2.bias loaded from backbone.body.layer3.5.bn2.bias of shape (256,) 2021-03-22 16:30:53,722 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.5.bn2.running_mean loaded from backbone.body.layer3.5.bn2.running_mean of shape (256,) 2021-03-22 16:30:53,722 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.5.bn2.running_var loaded from backbone.body.layer3.5.bn2.running_var of shape (256,) 2021-03-22 16:30:53,722 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.5.bn2.weight loaded from backbone.body.layer3.5.bn2.weight of shape (256,) 2021-03-22 16:30:53,722 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.5.bn3.bias loaded from backbone.body.layer3.5.bn3.bias of shape (1024,) 2021-03-22 16:30:53,722 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.5.bn3.running_mean loaded from backbone.body.layer3.5.bn3.running_mean of shape (1024,) 2021-03-22 16:30:53,722 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.5.bn3.running_var loaded from backbone.body.layer3.5.bn3.running_var of shape (1024,) 2021-03-22 16:30:53,722 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.5.bn3.weight loaded from backbone.body.layer3.5.bn3.weight of shape (1024,) 2021-03-22 16:30:53,722 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.5.conv1.weight loaded from backbone.body.layer3.5.conv1.weight of shape (256, 1024, 1, 1) 2021-03-22 16:30:53,722 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.5.conv2.weight loaded from backbone.body.layer3.5.conv2.weight of shape (256, 256, 3, 3) 2021-03-22 16:30:53,722 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer3.5.conv3.weight loaded from backbone.body.layer3.5.conv3.weight of shape (1024, 256, 1, 1) 2021-03-22 16:30:53,722 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.0.bn1.bias loaded from backbone.body.layer4.0.bn1.bias of shape (512,) 2021-03-22 16:30:53,722 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.0.bn1.running_mean loaded from backbone.body.layer4.0.bn1.running_mean of shape (512,) 2021-03-22 16:30:53,722 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.0.bn1.running_var loaded from backbone.body.layer4.0.bn1.running_var of shape (512,) 2021-03-22 16:30:53,722 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.0.bn1.weight loaded from backbone.body.layer4.0.bn1.weight of shape (512,) 2021-03-22 16:30:53,722 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.0.bn2.bias loaded from backbone.body.layer4.0.bn2.bias of shape (512,) 2021-03-22 16:30:53,722 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.0.bn2.running_mean loaded from backbone.body.layer4.0.bn2.running_mean of shape (512,) 2021-03-22 16:30:53,722 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.0.bn2.running_var loaded from backbone.body.layer4.0.bn2.running_var of shape (512,) 2021-03-22 16:30:53,722 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.0.bn2.weight loaded from backbone.body.layer4.0.bn2.weight of shape (512,) 2021-03-22 16:30:53,722 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.0.bn3.bias loaded from backbone.body.layer4.0.bn3.bias of shape (2048,) 2021-03-22 16:30:53,722 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.0.bn3.running_mean loaded from backbone.body.layer4.0.bn3.running_mean of shape (2048,) 2021-03-22 16:30:53,722 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.0.bn3.running_var loaded from backbone.body.layer4.0.bn3.running_var of shape (2048,) 2021-03-22 16:30:53,722 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.0.bn3.weight loaded from backbone.body.layer4.0.bn3.weight of shape (2048,) 2021-03-22 16:30:53,722 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.0.conv1.weight loaded from backbone.body.layer4.0.conv1.weight of shape (512, 1024, 1, 1) 2021-03-22 16:30:53,722 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.0.conv2.weight loaded from backbone.body.layer4.0.conv2.weight of shape (512, 512, 3, 3) 2021-03-22 16:30:53,722 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.0.conv3.weight loaded from backbone.body.layer4.0.conv3.weight of shape (2048, 512, 1, 1) 2021-03-22 16:30:53,722 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.0.downsample.0.weight loaded from backbone.body.layer4.0.downsample.0.weight of shape (2048, 1024, 1, 1) 2021-03-22 16:30:53,722 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.0.downsample.1.bias loaded from backbone.body.layer4.0.downsample.1.bias of shape (2048,) 2021-03-22 16:30:53,723 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.0.downsample.1.running_mean loaded from backbone.body.layer4.0.downsample.1.running_mean of shape (2048,) 2021-03-22 16:30:53,723 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.0.downsample.1.running_var loaded from backbone.body.layer4.0.downsample.1.running_var of shape (2048,) 2021-03-22 16:30:53,723 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.0.downsample.1.weight loaded from backbone.body.layer4.0.downsample.1.weight of shape (2048,) 2021-03-22 16:30:53,723 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.1.bn1.bias loaded from backbone.body.layer4.1.bn1.bias of shape (512,) 2021-03-22 16:30:53,723 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.1.bn1.running_mean loaded from backbone.body.layer4.1.bn1.running_mean of shape (512,) 2021-03-22 16:30:53,723 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.1.bn1.running_var loaded from backbone.body.layer4.1.bn1.running_var of shape (512,) 2021-03-22 16:30:53,723 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.1.bn1.weight loaded from backbone.body.layer4.1.bn1.weight of shape (512,) 2021-03-22 16:30:53,723 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.1.bn2.bias loaded from backbone.body.layer4.1.bn2.bias of shape (512,) 2021-03-22 16:30:53,723 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.1.bn2.running_mean loaded from backbone.body.layer4.1.bn2.running_mean of shape (512,) 2021-03-22 16:30:53,723 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.1.bn2.running_var loaded from backbone.body.layer4.1.bn2.running_var of shape (512,) 2021-03-22 16:30:53,723 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.1.bn2.weight loaded from backbone.body.layer4.1.bn2.weight of shape (512,) 2021-03-22 16:30:53,723 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.1.bn3.bias loaded from backbone.body.layer4.1.bn3.bias of shape (2048,) 2021-03-22 16:30:53,723 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.1.bn3.running_mean loaded from backbone.body.layer4.1.bn3.running_mean of shape (2048,) 2021-03-22 16:30:53,723 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.1.bn3.running_var loaded from backbone.body.layer4.1.bn3.running_var of shape (2048,) 2021-03-22 16:30:53,723 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.1.bn3.weight loaded from backbone.body.layer4.1.bn3.weight of shape (2048,) 2021-03-22 16:30:53,723 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.1.conv1.weight loaded from backbone.body.layer4.1.conv1.weight of shape (512, 2048, 1, 1) 2021-03-22 16:30:53,723 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.1.conv2.weight loaded from backbone.body.layer4.1.conv2.weight of shape (512, 512, 3, 3) 2021-03-22 16:30:53,723 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.1.conv3.weight loaded from backbone.body.layer4.1.conv3.weight of shape (2048, 512, 1, 1) 2021-03-22 16:30:53,723 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.2.bn1.bias loaded from backbone.body.layer4.2.bn1.bias of shape (512,) 2021-03-22 16:30:53,723 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.2.bn1.running_mean loaded from backbone.body.layer4.2.bn1.running_mean of shape (512,) 2021-03-22 16:30:53,723 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.2.bn1.running_var loaded from backbone.body.layer4.2.bn1.running_var of shape (512,) 2021-03-22 16:30:53,723 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.2.bn1.weight loaded from backbone.body.layer4.2.bn1.weight of shape (512,) 2021-03-22 16:30:53,723 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.2.bn2.bias loaded from backbone.body.layer4.2.bn2.bias of shape (512,) 2021-03-22 16:30:53,723 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.2.bn2.running_mean loaded from backbone.body.layer4.2.bn2.running_mean of shape (512,) 2021-03-22 16:30:53,723 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.2.bn2.running_var loaded from backbone.body.layer4.2.bn2.running_var of shape (512,) 2021-03-22 16:30:53,723 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.2.bn2.weight loaded from backbone.body.layer4.2.bn2.weight of shape (512,) 2021-03-22 16:30:53,723 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.2.bn3.bias loaded from backbone.body.layer4.2.bn3.bias of shape (2048,) 2021-03-22 16:30:53,724 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.2.bn3.running_mean loaded from backbone.body.layer4.2.bn3.running_mean of shape (2048,) 2021-03-22 16:30:53,724 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.2.bn3.running_var loaded from backbone.body.layer4.2.bn3.running_var of shape (2048,) 2021-03-22 16:30:53,724 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.2.bn3.weight loaded from backbone.body.layer4.2.bn3.weight of shape (2048,) 2021-03-22 16:30:53,724 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.2.conv1.weight loaded from backbone.body.layer4.2.conv1.weight of shape (512, 2048, 1, 1) 2021-03-22 16:30:53,724 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.2.conv2.weight loaded from backbone.body.layer4.2.conv2.weight of shape (512, 512, 3, 3) 2021-03-22 16:30:53,724 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.layer4.2.conv3.weight loaded from backbone.body.layer4.2.conv3.weight of shape (2048, 512, 1, 1) 2021-03-22 16:30:53,724 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.stem.bn1.bias loaded from backbone.body.stem.bn1.bias of shape (64,) 2021-03-22 16:30:53,724 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.stem.bn1.running_mean loaded from backbone.body.stem.bn1.running_mean of shape (64,) 2021-03-22 16:30:53,724 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.stem.bn1.running_var loaded from backbone.body.stem.bn1.running_var of shape (64,) 2021-03-22 16:30:53,724 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.stem.bn1.weight loaded from backbone.body.stem.bn1.weight of shape (64,) 2021-03-22 16:30:53,724 maskrcnn_benchmark.utils.model_serialization INFO: backbone.body.stem.conv1.weight loaded from backbone.body.stem.conv1.weight of shape (64, 3, 7, 7) 2021-03-22 16:30:53,724 maskrcnn_benchmark.utils.model_serialization INFO: backbone.fpn.fpn_inner1.bias loaded from backbone.fpn.fpn_inner1.bias of shape (256,) 2021-03-22 16:30:53,724 maskrcnn_benchmark.utils.model_serialization INFO: backbone.fpn.fpn_inner1.weight loaded from backbone.fpn.fpn_inner1.weight of shape (256, 256, 1, 1) 2021-03-22 16:30:53,724 maskrcnn_benchmark.utils.model_serialization INFO: backbone.fpn.fpn_inner2.bias loaded from backbone.fpn.fpn_inner2.bias of shape (256,) 2021-03-22 16:30:53,724 maskrcnn_benchmark.utils.model_serialization INFO: backbone.fpn.fpn_inner2.weight loaded from backbone.fpn.fpn_inner2.weight of shape (256, 512, 1, 1) 2021-03-22 16:30:53,724 maskrcnn_benchmark.utils.model_serialization INFO: backbone.fpn.fpn_inner3.bias loaded from backbone.fpn.fpn_inner3.bias of shape (256,) 2021-03-22 16:30:53,724 maskrcnn_benchmark.utils.model_serialization INFO: backbone.fpn.fpn_inner3.weight loaded from backbone.fpn.fpn_inner3.weight of shape (256, 1024, 1, 1) 2021-03-22 16:30:53,724 maskrcnn_benchmark.utils.model_serialization INFO: backbone.fpn.fpn_inner4.bias loaded from backbone.fpn.fpn_inner4.bias of shape (256,) 2021-03-22 16:30:53,724 maskrcnn_benchmark.utils.model_serialization INFO: backbone.fpn.fpn_inner4.weight loaded from backbone.fpn.fpn_inner4.weight of shape (256, 2048, 1, 1) 2021-03-22 16:30:53,724 maskrcnn_benchmark.utils.model_serialization INFO: backbone.fpn.fpn_layer1.bias loaded from backbone.fpn.fpn_layer1.bias of shape (256,) 2021-03-22 16:30:53,724 maskrcnn_benchmark.utils.model_serialization INFO: backbone.fpn.fpn_layer1.weight loaded from backbone.fpn.fpn_layer1.weight of shape (256, 256, 3, 3) 2021-03-22 16:30:53,724 maskrcnn_benchmark.utils.model_serialization INFO: backbone.fpn.fpn_layer2.bias loaded from backbone.fpn.fpn_layer2.bias of shape (256,) 2021-03-22 16:30:53,724 maskrcnn_benchmark.utils.model_serialization INFO: backbone.fpn.fpn_layer2.weight loaded from backbone.fpn.fpn_layer2.weight of shape (256, 256, 3, 3) 2021-03-22 16:30:53,724 maskrcnn_benchmark.utils.model_serialization INFO: backbone.fpn.fpn_layer3.bias loaded from backbone.fpn.fpn_layer3.bias of shape (256,) 2021-03-22 16:30:53,724 maskrcnn_benchmark.utils.model_serialization INFO: backbone.fpn.fpn_layer3.weight loaded from backbone.fpn.fpn_layer3.weight of shape (256, 256, 3, 3) 2021-03-22 16:30:53,724 maskrcnn_benchmark.utils.model_serialization INFO: backbone.fpn.fpn_layer4.bias loaded from backbone.fpn.fpn_layer4.bias of shape (256,) 2021-03-22 16:30:53,724 maskrcnn_benchmark.utils.model_serialization INFO: backbone.fpn.fpn_layer4.weight loaded from backbone.fpn.fpn_layer4.weight of shape (256, 256, 3, 3) 2021-03-22 16:30:53,724 maskrcnn_benchmark.utils.model_serialization INFO: roi_heads.box.feature_extractor.fc6.bias loaded from roi_heads.box.feature_extractor.fc6.bias of shape (1024,) 2021-03-22 16:30:53,725 maskrcnn_benchmark.utils.model_serialization INFO: roi_heads.box.feature_extractor.fc6.weight loaded from roi_heads.box.feature_extractor.fc6.weight of shape (1024, 12544) 2021-03-22 16:30:53,725 maskrcnn_benchmark.utils.model_serialization INFO: roi_heads.box.feature_extractor.fc7.bias loaded from roi_heads.box.feature_extractor.fc7.bias of shape (1024,) 2021-03-22 16:30:53,725 maskrcnn_benchmark.utils.model_serialization INFO: roi_heads.box.feature_extractor.fc7.weight loaded from roi_heads.box.feature_extractor.fc7.weight of shape (1024, 1024) 2021-03-22 16:30:53,725 maskrcnn_benchmark.utils.model_serialization INFO: rpn.anchor_generator.cell_anchors.0 loaded from rpn.anchor_generator.cell_anchors.0 of shape (3, 4) 2021-03-22 16:30:53,725 maskrcnn_benchmark.utils.model_serialization INFO: rpn.anchor_generator.cell_anchors.1 loaded from rpn.anchor_generator.cell_anchors.1 of shape (3, 4) 2021-03-22 16:30:53,725 maskrcnn_benchmark.utils.model_serialization INFO: rpn.anchor_generator.cell_anchors.2 loaded from rpn.anchor_generator.cell_anchors.2 of shape (3, 4) 2021-03-22 16:30:53,725 maskrcnn_benchmark.utils.model_serialization INFO: rpn.anchor_generator.cell_anchors.3 loaded from rpn.anchor_generator.cell_anchors.3 of shape (3, 4) 2021-03-22 16:30:53,725 maskrcnn_benchmark.utils.model_serialization INFO: rpn.anchor_generator.cell_anchors.4 loaded from rpn.anchor_generator.cell_anchors.4 of shape (3, 4) 2021-03-22 16:30:53,725 maskrcnn_benchmark.utils.model_serialization INFO: rpn.head.bbox_pred.bias loaded from rpn.head.bbox_pred.bias of shape (12,) 2021-03-22 16:30:53,725 maskrcnn_benchmark.utils.model_serialization INFO: rpn.head.bbox_pred.weight loaded from rpn.head.bbox_pred.weight of shape (12, 256, 1, 1) 2021-03-22 16:30:53,725 maskrcnn_benchmark.utils.model_serialization INFO: rpn.head.cls_logits.bias loaded from rpn.head.cls_logits.bias of shape (3,) 2021-03-22 16:30:53,725 maskrcnn_benchmark.utils.model_serialization INFO: rpn.head.cls_logits.weight loaded from rpn.head.cls_logits.weight of shape (3, 256, 1, 1) 2021-03-22 16:30:53,725 maskrcnn_benchmark.utils.model_serialization INFO: rpn.head.conv.bias loaded from rpn.head.conv.bias of shape (256,) 2021-03-22 16:30:53,725 maskrcnn_benchmark.utils.model_serialization INFO: rpn.head.conv.weight loaded from rpn.head.conv.weight of shape (256, 256, 3, 3) loading annotations into memory... Traceback (most recent call last): File "tools/train_net.py", line 201, in main() File "tools/train_net.py", line 194, in main model = train(cfg, args.local_rank, args.distributed) File "tools/train_net.py", line 72, in train start_iter=arguments["iteration"], File "/home/delta/anaconda3/envs/p2_trainer/lib/python3.6/site-packages/maskrcnn_benchmark-0.1-py3.6-linux-x86_64.egg/maskrcnn_benchmark/data/build.py", line 156, in make_data_loader datasets = build_dataset(dataset_list, transforms, DatasetCatalog, is_train or is_for_period) File "/home/delta/anaconda3/envs/p2_trainer/lib/python3.6/site-packages/maskrcnn_benchmark-0.1-py3.6-linux-x86_64.egg/maskrcnn_benchmark/data/build.py", line 45, in build_dataset dataset = factory(**args) File "/home/delta/anaconda3/envs/p2_trainer/lib/python3.6/site-packages/maskrcnn_benchmark-0.1-py3.6-linux-x86_64.egg/maskrcnn_benchmark/data/datasets/coco.py", line 43, in init super(COCODataset, self).init(root, ann_file) File "/home/delta/anaconda3/envs/p2_trainer/lib/python3.6/site-packages/torchvision/datasets/coco.py", line 98, in init self.coco = COCO(annFile) File "/home/delta/anaconda3/envs/p2_trainer/lib/python3.6/site-packages/pycocotools-2.0-py3.6-linux-x86_64.egg/pycocotools/coco.py", line 84, in init dataset = json.load(open(annotation_file, 'r')) FileNotFoundError: [Errno 2] No such file or directory: 'datasets/custom_dataset/train_dataset/annotations.json' cp: cannot stat 'maskrcnn-benchmark/weights/custom/model_0003000.pth': No such file or directory [p2_exporter] - FOUND. Skipping installation. cp: cannot stat 'trained_models/trained.pth': No such file or directory Traceback (most recent call last): File "demo/export_to_onnx.py", line 58, in min_image_size=480, File "/home/delta/epd_ros2_ws/src/easy_perception_deployment/easy_perception_deployment/gui/trainer/P2TrainFarm/p2exporter/demo/predictor.py", line 148, in init = checkpointer.load(cfg.MODEL.WEIGHT) File "/home/delta/epd_ros2_ws/src/easy_perception_deployment/easy_perception_deployment/gui/trainer/P2TrainFarm/p2_exporter/maskrcnn_benchmark/utils/checkpoint.py", line 61, in load checkpoint = self._load_file(f) File "/home/delta/epd_ros2_ws/src/easy_perception_deployment/easy_perception_deployment/gui/trainer/P2TrainFarm/p2_exporter/maskrcnn_benchmark/utils/checkpoint.py", line 136, in _load_file loaded = super(DetectronCheckpointer, self)._load_file(f) File "/home/delta/epd_ros2_ws/src/easy_perception_deployment/easy_perception_deployment/gui/trainer/P2TrainFarm/p2_exporter/maskrcnn_benchmark/utils/checkpoint.py", line 95, in _load_file return torch.load(f, map_location=torch.device("cpu")) File "/home/delta/anaconda3/envs/p2_exporter/lib/python3.6/site-packages/torch/serialization.py", line 381, in load f = open(f, 'rb') FileNotFoundError: [Errno 2] No such file or directory: 'weights/custom/trained.pth' Traceback (most recent call last): File "demo/remove_initializer.py", line 61, in remove_initializer_from_input() File "demo/remove_initializer.py", line 40, in remove_initializer_from_input model = onnx.load(args.input) File "/home/delta/anaconda3/envs/p2_exporter/lib/python3.6/site-packages/onnx/init.py", line 118, in load_model s = _load_bytes(f) File "/home/delta/anaconda3/envs/p2_exporter/lib/python3.6/site-packages/onnx/init.py", line 32, in _load_bytes with open(cast(Text, f), 'rb') as readable: FileNotFoundError: [Errno 2] No such file or directory: 'weights/custom.onnx' cp: cannot stat 'p2_exporter/weights/exported_p2.onnx': No such file or directory Model trained and exported successfully. Please find your model at the following directory -> [/home/delta/epd_ros2_ws/src/easy_perception_deployment/easy_perception_deployment/gui/trainer/P3TrainFarm/trained_models/fasterrcnn-2021-03-22.onnx]

cardboardcode commented 3 years ago

Noted. Will look into it once I have the bandwidth.

cardboardcode commented 3 years ago

Hi @Srikiranrao ,

As identified by your findings and the terminal output attached above, the cause of the issue is due to the labelled dataset not being properly arranged in an accepted format.

File "/home/delta/anaconda3/envs/p2_trainer/lib/python3.6/site-packages/pycocotools-2.0-py3.6-linux-x86_64.egg/pycocotools/coco.py", line 84, in init
dataset = json.load(open(annotation_file, 'r'))
## **FileNotFoundError: [Errno 2] No such file or directory: 'datasets/custom_dataset/train_dataset/annotations.json'**
cp: cannot stat 'maskrcnn-benchmark/weights/custom/model_0003000.pth': No such file or directory
[p2_exporter] - FOUND. Skipping installation.
cp: cannot stat 'trained_models/trained.pth': No such file or directory

Below attached is a 2 .gif animations showing how your labelled dataset should look like in your file manager before the training can start properly.

Labelled Dataset but not COCO-formatted

Please ensure your labelled image dataset has the same structure as what is shown below: labelled_not_coco-formated

Generate COCO-formatted Labelled Dataset

generate_dataset

Labelled Dataset and COCO-formatted

Please ensure your labelled image dataset has the same structure as what is shown below after you have selected the above folder via the Generate Dataset GUI button. labelled_coco-formated

Srikiranrao commented 3 years ago

HI @cardboardcode , Thanks for your response. I was successful in generating the Dataset from reference above.

While proceeding to next step -> "Choose Dataset" option from the GUI app, I faced an issue "Dataset not properly restructured. Please restructure Dataset" . May I know how to resolve this ?

issue2021-03-30 10-45

cardboardcode commented 3 years ago

@Srikiranrao Don't worry about that error. The attached .gif you have provided showed that your Dataset is correctly restructured.

Please run the following commands to update your current iteration of EPD and this error should go away.

git pull 
# Build the easy_perception_deployment package again.
Srikiranrao commented 3 years ago

@cardboardcode Thanks, I did the git pull and built the package. when I choose "Train" option I received following issue in screenshot. It's looking for "maskrcnn-benchmark" file which is not available in that location.

Screenshot from 2021-04-01 09-59-30

Also do you need me to create a seperate Issue for this as the above listed Issue was resolved.

cardboardcode commented 3 years ago

Please use the commands below to do the following:

Rebuild the p3_trainer & p3_exporter Anaconda environment so that it will build the maskrcnn-benchmark folder.

conda env remove -n p3_trainer
conda env remove -n p3_exporter
# Configure your Train option in the GUI and run Train again.

The maskrcnn-benchmark folder should appear in the following directory: easy_perception_deployment/gui/trainer/P2TrainFarm since Precision Level 2 has been selected for this issue.

cardboardcode commented 3 years ago

Based on internal discussion, it has been verified that the issue has been resolved and the .onnx model file is generated.

Closing issue.

Srikiranrao commented 3 years ago

@cardboardcode , Thanks for the support.