search

open-mmlab / mmdetection

OpenMMLab Detection Toolbox and Benchmark
https://mmdetection.readthedocs.io
Apache License 2.0
29.14k stars 9.39k forks source link

training is so slow #5237

Closed olfa-koubaa closed 3 years ago

olfa-koubaa commented 3 years ago

I tried training the mask rcnn with both detectron2 and mmdetection and the training speed gap is huge

mmdetection offers a lot of preimplemented models that I want to run my tests with but I'm struggling with the fact that is so slow

I found some people who encountered the same problem but it's not clear as to what's causing it or what is the solution

RangiLyu commented 3 years ago

Hi, can you provide more details e.g. training config, log files, your environment, and the training time difference to help us find out the reason?

tehkillerbee commented 3 years ago

Are you sure you are loading a checkpoint to speed up training? How many images are you using when training?

olfa-koubaa commented 3 years ago

this is the log for when I trained with yolact because I lost the one with mask rcnn ( but I will reproduce it and put it later )

it was basically the same speed for the two of them it took hours to run 50 epochs

2021-05-26 10:38:00,312 - mmdet - INFO - Environment info:

sys.platform: linux Python: 3.7.10 (default, May 3 2021, 02:48:31) [GCC 7.5.0] CUDA available: True GPU 0: Tesla P100-PCIE-16GB CUDA_HOME: /usr/local/cuda NVCC: Build cuda_11.0_bu.TC445_37.28845127_0 GCC: gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0 PyTorch: 1.8.1+cu101 PyTorch compiling details: PyTorch built with:

TorchVision: 0.9.1+cu101 OpenCV: 4.1.2 MMCV: 1.3.4 MMCV Compiler: GCC 7.5 MMCV CUDA Compiler: 11.0 MMDetection: 2.12.0+5f94442

2021-05-26 10:38:00,703 - mmdet - INFO - Distributed training: False 2021-05-26 10:38:01,084 - mmdet - INFO - Config: checkpoint_config = dict(interval=1) log_config = dict(interval=50, hooks=[dict(type='TextLoggerHook')]) custom_hooks = [dict(type='NumClassCheckHook')] dist_params = dict(backend='nccl') log_level = 'INFO' load_from = '/content/mmdetection/checkpoints/yolact_r50_1x8_coco_20200908-f38d58df.pth' resume_from = None workflow = [('train', 1)] img_size = 550 model = dict( type='YOLACT', pretrained='torchvision://resnet50', backbone=dict( type='ResNet', depth=50, num_stages=4, out_indices=(0, 1, 2, 3), frozen_stages=-1, norm_cfg=dict(type='BN', requires_grad=True), norm_eval=False, zero_init_residual=False, style='pytorch'), neck=dict( type='FPN', in_channels=[256, 512, 1024, 2048], out_channels=256, start_level=1, add_extra_convs='on_input', num_outs=5, upsample_cfg=dict(mode='bilinear')), bbox_head=dict( type='YOLACTHead', num_classes=1, in_channels=256, feat_channels=256, anchor_generator=dict( type='AnchorGenerator', octave_base_scale=3, scales_per_octave=1, base_sizes=[8, 16, 32, 64, 128], ratios=[0.5, 1.0, 2.0], strides=[ 7.971014492753623, 15.714285714285714, 30.555555555555557, 61.111111111111114, 110.0 ], centers=[(3.9855072463768115, 3.9855072463768115), (7.857142857142857, 7.857142857142857), (15.277777777777779, 15.277777777777779), (30.555555555555557, 30.555555555555557), (55.0, 55.0)]), bbox_coder=dict( type='DeltaXYWHBBoxCoder', target_means=[0.0, 0.0, 0.0, 0.0], target_stds=[0.1, 0.1, 0.2, 0.2]), loss_cls=dict( type='CrossEntropyLoss', use_sigmoid=False, reduction='none', loss_weight=1.0), loss_bbox=dict(type='SmoothL1Loss', beta=1.0, loss_weight=1.5), num_head_convs=1, num_protos=32, use_ohem=True), mask_head=dict( type='YOLACTProtonet', in_channels=256, num_protos=32, num_classes=1, max_masks_to_train=100, loss_mask_weight=6.125), segm_head=dict( type='YOLACTSegmHead', num_classes=1, in_channels=256, loss_segm=dict( type='CrossEntropyLoss', use_sigmoid=True, loss_weight=1.0)), train_cfg=dict( assigner=dict( type='MaxIoUAssigner', pos_iou_thr=0.5, neg_iou_thr=0.4, min_pos_iou=0.0, ignore_iof_thr=-1, gt_max_assign_all=False), allowed_border=-1, pos_weight=-1, neg_pos_ratio=3, debug=False), test_cfg=dict( nms_pre=1000, min_bbox_size=0, score_thr=0.05, iou_thr=0.5, top_k=200, max_per_img=100)) dataset_type = 'COCODataset' data_root = 'data/coco/' img_norm_cfg = dict( mean=[123.68, 116.78, 103.94], std=[58.4, 57.12, 57.38], to_rgb=True) train_pipeline = [ dict(type='LoadImageFromFile', to_float32=True), dict(type='LoadAnnotations', with_bbox=True, with_mask=True), dict(type='FilterAnnotations', min_gt_bbox_wh=(4.0, 4.0)), dict( type='PhotoMetricDistortion', brightness_delta=32, contrast_range=(0.5, 1.5), saturation_range=(0.5, 1.5), hue_delta=18), dict( type='Expand', mean=[123.68, 116.78, 103.94], to_rgb=True, ratio_range=(1, 4)), dict( type='MinIoURandomCrop', min_ious=(0.1, 0.3, 0.5, 0.7, 0.9), min_crop_size=0.3), dict(type='Resize', img_scale=(550, 550), keep_ratio=False), dict( type='Normalize', mean=[123.68, 116.78, 103.94], std=[58.4, 57.12, 57.38], to_rgb=True), dict(type='RandomFlip', flip_ratio=0.5), dict(type='DefaultFormatBundle'), dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels', 'gt_masks']) ] test_pipeline = [ dict(type='LoadImageFromFile'), dict( type='MultiScaleFlipAug', img_scale=(550, 550), flip=False, transforms=[ dict(type='Resize', keep_ratio=False), dict( type='Normalize', mean=[123.68, 116.78, 103.94], std=[58.4, 57.12, 57.38], to_rgb=True), dict(type='ImageToTensor', keys=['img']), dict(type='Collect', keys=['img']) ]) ] data = dict( samples_per_gpu=8, workers_per_gpu=4, train=dict( type='CocoDataset', ann_file='/content/train_coco.json', img_prefix='/content/train_data/', pipeline=[ dict(type='LoadImageFromFile', to_float32=True), dict(type='LoadAnnotations', with_bbox=True, with_mask=True), dict(type='FilterAnnotations', min_gt_bbox_wh=(4.0, 4.0)), dict( type='PhotoMetricDistortion', brightness_delta=32, contrast_range=(0.5, 1.5), saturation_range=(0.5, 1.5), hue_delta=18), dict( type='Expand', mean=[123.68, 116.78, 103.94], to_rgb=True, ratio_range=(1, 4)), dict( type='MinIoURandomCrop', min_ious=(0.1, 0.3, 0.5, 0.7, 0.9), min_crop_size=0.3), dict(type='Resize', img_scale=(550, 550), keep_ratio=False), dict( type='Normalize', mean=[123.68, 116.78, 103.94], std=[58.4, 57.12, 57.38], to_rgb=True), dict(type='RandomFlip', flip_ratio=0.5), dict(type='DefaultFormatBundle'), dict( type='Collect', keys=['img', 'gt_bboxes', 'gt_labels', 'gt_masks']) ], classes=('lichen', )), val=dict( type='CocoDataset', ann_file='/content/val_coco.json', img_prefix='/content/val_data', pipeline=[ dict(type='LoadImageFromFile'), dict( type='MultiScaleFlipAug', img_scale=(550, 550), flip=False, transforms=[ dict(type='Resize', keep_ratio=False), dict( type='Normalize', mean=[123.68, 116.78, 103.94], std=[58.4, 57.12, 57.38], to_rgb=True), dict(type='ImageToTensor', keys=['img']), dict(type='Collect', keys=['img']) ]) ], classes=('lichen', )), test=dict( type='CocoDataset', ann_file='/content/test_coco.json', img_prefix='/content/test_data', pipeline=[ dict(type='LoadImageFromFile'), dict( type='MultiScaleFlipAug', img_scale=(550, 550), flip=False, transforms=[ dict(type='Resize', keep_ratio=False), dict( type='Normalize', mean=[123.68, 116.78, 103.94], std=[58.4, 57.12, 57.38], to_rgb=True), dict(type='ImageToTensor', keys=['img']), dict(type='Collect', keys=['img']) ]) ], classes=('lichen', ))) optimizer = dict(type='SGD', lr=0.001, momentum=0.9, weight_decay=0.0005) optimizer_config = dict() lr_config = dict( policy='step', warmup='linear', warmup_iters=500, warmup_ratio=0.1, step=[20, 42, 49, 52]) runner = dict(type='EpochBasedRunner', max_epochs=50) cudnn_benchmark = True evaluation = dict(metric=['bbox', 'segm']) classes = ('lichen', ) work_dir = './work_dirs/yolact' gpu_ids = range(0, 1)

2021-05-26 10:38:08,507 - mmdet - INFO - load checkpoint from /content/mmdetection/checkpoints/yolact_r50_1x8_coco_20200908-f38d58df.pth 2021-05-26 10:38:08,507 - mmdet - INFO - Use load_from_local loader 2021-05-26 10:38:08,651 - mmdet - WARNING - The model and loaded state dict do not match exactly

size mismatch for bbox_head.conv_cls.weight: copying a param with shape torch.Size([243, 256, 3, 3]) from checkpoint, the shape in current model is torch.Size([6, 256, 3, 3]). size mismatch for bbox_head.conv_cls.bias: copying a param with shape torch.Size([243]) from checkpoint, the shape in current model is torch.Size([6]). size mismatch for segm_head.segm_conv.weight: copying a param with shape torch.Size([80, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([1, 256, 1, 1]). size mismatch for segm_head.segm_conv.bias: copying a param with shape torch.Size([80]) from checkpoint, the shape in current model is torch.Size([1]). 2021-05-26 10:38:08,654 - mmdet - INFO - Start running, host: root@331a314d8e03, work_dir: /content/mmdetection/work_dirs/yolact 2021-05-26 10:38:08,654 - mmdet - INFO - workflow: [('train', 1)], max: 50 epochs 2021-05-26 10:41:28,795 - mmdet - INFO - Epoch [1][50/124] lr: 1.882e-04, eta: 6:50:05, time: 4.001, data_time: 3.438, memory: 11791, loss_cls: 3.4926, loss_bbox: 1.5993, loss_segm: 0.4193, loss_mask: 2.8211, loss: 8.3324 2021-05-26 10:45:05,037 - mmdet - INFO - Epoch [1][100/124] lr: 2.782e-04, eta: 7:03:13, time: 4.325, data_time: 3.830, memory: 11791, loss_cls: 1.7552, loss_bbox: 0.8845, loss_segm: 0.1841, loss_mask: 2.0905, loss: 4.9143 2021-05-26 10:48:37,596 - mmdet - INFO - Evaluating bbox... 2021-05-26 10:48:37,965 - mmdet - INFO - Evaluating segm... 2021-05-26 10:48:38,813 - mmdet - INFO - Saving checkpoint at 1 epochs 2021-05-26 10:48:39,629 - mmdet - INFO - Exp name: yolact.py 2021-05-26 10:48:39,630 - mmdet - INFO - Epoch(val) [1][206] bbox_mAP: 0.4560, bbox_mAP_50: 0.6250, bbox_mAP_75: 0.5320, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3710, bbox_mAP_l: 0.4770, bbox_mAP_copypaste: 0.456 0.625 0.532 -1.000 0.371 0.477, segm_mAP: 0.3960, segm_mAP_50: 0.5900, segm_mAP_75: 0.4150, segm_mAP_s: -1.0000, segm_mAP_m: 0.1070, segm_mAP_l: 0.4370, segm_mAP_copypaste: 0.396 0.590 0.415 -1.000 0.107 0.437 2021-05-26 10:52:08,261 - mmdet - INFO - Epoch [2][50/124] lr: 4.114e-04, eta: 6:00:38, time: 4.171, data_time: 3.676, memory: 11791, loss_cls: 1.0475, loss_bbox: 0.5576, loss_segm: 0.1204, loss_mask: 1.6205, loss: 3.3460 2021-05-26 10:55:46,794 - mmdet - INFO - Epoch [2][100/124] lr: 5.014e-04, eta: 6:14:59, time: 4.371, data_time: 3.879, memory: 11791, loss_cls: 0.8611, loss_bbox: 0.4973, loss_segm: 0.1047, loss_mask: 1.5544, loss: 3.0175 2021-05-26 10:58:28,123 - mmdet - INFO - Evaluating bbox... 2021-05-26 10:58:28,250 - mmdet - INFO - Evaluating segm... 2021-05-26 10:58:28,440 - mmdet - INFO - Saving checkpoint at 2 epochs 2021-05-26 10:58:29,104 - mmdet - INFO - Exp name: yolact.py 2021-05-26 10:58:29,104 - mmdet - INFO - Epoch(val) [2][206] bbox_mAP: 0.5590, bbox_mAP_50: 0.8040, bbox_mAP_75: 0.6740, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3370, bbox_mAP_l: 0.6000, bbox_mAP_copypaste: 0.559 0.804 0.674 -1.000 0.337 0.600, segm_mAP: 0.5180, segm_mAP_50: 0.7680, segm_mAP_75: 0.5500, segm_mAP_s: -1.0000, segm_mAP_m: 0.1290, segm_mAP_l: 0.5780, segm_mAP_copypaste: 0.518 0.768 0.550 -1.000 0.129 0.578 2021-05-26 11:01:42,241 - mmdet - INFO - Epoch [3][50/124] lr: 6.346e-04, eta: 5:42:05, time: 3.861, data_time: 3.352, memory: 11791, loss_cls: 0.6860, loss_bbox: 0.4495, loss_segm: 0.0918, loss_mask: 1.4871, loss: 2.7143 2021-05-26 11:05:14,425 - mmdet - INFO - Epoch [3][100/124] lr: 7.246e-04, eta: 5:49:56, time: 4.244, data_time: 3.747, memory: 11791, loss_cls: 0.6532, loss_bbox: 0.3891, loss_segm: 0.0877, loss_mask: 1.4312, loss: 2.5612 2021-05-26 11:07:46,198 - mmdet - INFO - Evaluating bbox... 2021-05-26 11:07:46,330 - mmdet - INFO - Evaluating segm... 2021-05-26 11:07:46,526 - mmdet - INFO - Saving checkpoint at 3 epochs 2021-05-26 11:07:47,193 - mmdet - INFO - Exp name: yolact.py 2021-05-26 11:07:47,193 - mmdet - INFO - Epoch(val) [3][206] bbox_mAP: 0.5660, bbox_mAP_50: 0.7670, bbox_mAP_75: 0.6560, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.4050, bbox_mAP_l: 0.6010, bbox_mAP_copypaste: 0.566 0.767 0.656 -1.000 0.405 0.601, segm_mAP: 0.4820, segm_mAP_50: 0.7290, segm_mAP_75: 0.5060, segm_mAP_s: -1.0000, segm_mAP_m: 0.1350, segm_mAP_l: 0.5350, segm_mAP_copypaste: 0.482 0.729 0.506 -1.000 0.135 0.535 2021-05-26 11:11:22,109 - mmdet - INFO - Epoch [4][50/124] lr: 8.578e-04, eta: 5:33:56, time: 4.296, data_time: 3.806, memory: 11791, loss_cls: 0.6662, loss_bbox: 0.4357, loss_segm: 0.0896, loss_mask: 1.4335, loss: 2.6250 2021-05-26 11:14:55,639 - mmdet - INFO - Epoch [4][100/124] lr: 9.478e-04, eta: 5:39:10, time: 4.270, data_time: 3.774, memory: 11791, loss_cls: 0.5912, loss_bbox: 0.3734, loss_segm: 0.0778, loss_mask: 1.3431, loss: 2.3855 2021-05-26 11:17:37,407 - mmdet - INFO - Evaluating bbox... 2021-05-26 11:17:37,555 - mmdet - INFO - Evaluating segm... 2021-05-26 11:17:37,749 - mmdet - INFO - Saving checkpoint at 4 epochs 2021-05-26 11:17:38,431 - mmdet - INFO - Exp name: yolact.py 2021-05-26 11:17:38,431 - mmdet - INFO - Epoch(val) [4][206] bbox_mAP: 0.5220, bbox_mAP_50: 0.7310, bbox_mAP_75: 0.6240, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.4260, bbox_mAP_l: 0.5510, bbox_mAP_copypaste: 0.522 0.731 0.624 -1.000 0.426 0.551, segm_mAP: 0.4000, segm_mAP_50: 0.7030, segm_mAP_75: 0.3570, segm_mAP_s: -1.0000, segm_mAP_m: 0.1590, segm_mAP_l: 0.4410, segm_mAP_copypaste: 0.400 0.703 0.357 -1.000 0.159 0.441 2021-05-26 11:20:52,867 - mmdet - INFO - Epoch [5][50/124] lr: 1.000e-03, eta: 5:22:57, time: 3.887, data_time: 3.381, memory: 11791, loss_cls: 0.5487, loss_bbox: 0.3835, loss_segm: 0.0741, loss_mask: 1.2979, loss: 2.3042 2021-05-26 11:24:21,486 - mmdet - INFO - Epoch [5][100/124] lr: 1.000e-03, eta: 5:25:56, time: 4.172, data_time: 3.681, memory: 11791, loss_cls: 0.4589, loss_bbox: 0.3209, loss_segm: 0.0755, loss_mask: 1.2353, loss: 2.0905 2021-05-26 11:27:07,876 - mmdet - INFO - Evaluating bbox... 2021-05-26 11:27:08,015 - mmdet - INFO - Evaluating segm... 2021-05-26 11:27:08,215 - mmdet - INFO - Saving checkpoint at 5 epochs 2021-05-26 11:27:08,938 - mmdet - INFO - Exp name: yolact.py 2021-05-26 11:27:08,938 - mmdet - INFO - Epoch(val) [5][206] bbox_mAP: 0.6080, bbox_mAP_50: 0.8420, bbox_mAP_75: 0.7250, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.4600, bbox_mAP_l: 0.6460, bbox_mAP_copypaste: 0.608 0.842 0.725 -1.000 0.460 0.646, segm_mAP: 0.5040, segm_mAP_50: 0.7850, segm_mAP_75: 0.5190, segm_mAP_s: -1.0000, segm_mAP_m: 0.1130, segm_mAP_l: 0.5650, segm_mAP_copypaste: 0.504 0.785 0.519 -1.000 0.113 0.565 2021-05-26 11:30:40,107 - mmdet - INFO - Epoch [6][50/124] lr: 1.000e-03, eta: 5:15:08, time: 4.221, data_time: 3.734, memory: 11791, loss_cls: 0.4587, loss_bbox: 0.3544, loss_segm: 0.0714, loss_mask: 1.2498, loss: 2.1343 2021-05-26 11:34:12,510 - mmdet - INFO - Epoch [6][100/124] lr: 1.000e-03, eta: 5:17:33, time: 4.248, data_time: 3.746, memory: 11791, loss_cls: 0.4676, loss_bbox: 0.3030, loss_segm: 0.0775, loss_mask: 1.2155, loss: 2.0635 2021-05-26 11:36:57,119 - mmdet - INFO - Evaluating bbox... 2021-05-26 11:36:57,259 - mmdet - INFO - Evaluating segm... 2021-05-26 11:36:57,437 - mmdet - INFO - Saving checkpoint at 6 epochs 2021-05-26 11:36:58,114 - mmdet - INFO - Exp name: yolact.py 2021-05-26 11:36:58,114 - mmdet - INFO - Epoch(val) [6][206] bbox_mAP: 0.5240, bbox_mAP_50: 0.7320, bbox_mAP_75: 0.6120, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3860, bbox_mAP_l: 0.5530, bbox_mAP_copypaste: 0.524 0.732 0.612 -1.000 0.386 0.553, segm_mAP: 0.4170, segm_mAP_50: 0.6700, segm_mAP_75: 0.4040, segm_mAP_s: -1.0000, segm_mAP_m: 0.0780, segm_mAP_l: 0.4660, segm_mAP_copypaste: 0.417 0.670 0.404 -1.000 0.078 0.466 2021-05-26 11:40:59,434 - mmdet - INFO - Epoch [7][50/124] lr: 1.000e-03, eta: 5:11:26, time: 4.824, data_time: 4.323, memory: 11791, loss_cls: 0.5003, loss_bbox: 0.3094, loss_segm: 0.0693, loss_mask: 1.2564, loss: 2.1354 2021-05-26 11:44:28,534 - mmdet - INFO - Epoch [7][100/124] lr: 1.000e-03, eta: 5:12:23, time: 4.182, data_time: 3.684, memory: 11791, loss_cls: 0.4061, loss_bbox: 0.2881, loss_segm: 0.0659, loss_mask: 1.1540, loss: 1.9141 2021-05-26 11:47:07,180 - mmdet - INFO - Evaluating bbox... 2021-05-26 11:47:07,328 - mmdet - INFO - Evaluating segm... 2021-05-26 11:47:07,516 - mmdet - INFO - Saving checkpoint at 7 epochs 2021-05-26 11:47:08,183 - mmdet - INFO - Exp name: yolact.py 2021-05-26 11:47:08,183 - mmdet - INFO - Epoch(val) [7][206] bbox_mAP: 0.5070, bbox_mAP_50: 0.6950, bbox_mAP_75: 0.5830, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3470, bbox_mAP_l: 0.5400, bbox_mAP_copypaste: 0.507 0.695 0.583 -1.000 0.347 0.540, segm_mAP: 0.3990, segm_mAP_50: 0.6460, segm_mAP_75: 0.4200, segm_mAP_s: -1.0000, segm_mAP_m: 0.0930, segm_mAP_l: 0.4440, segm_mAP_copypaste: 0.399 0.646 0.420 -1.000 0.093 0.444 2021-05-26 11:50:45,882 - mmdet - INFO - Epoch [8][50/124] lr: 1.000e-03, eta: 5:04:06, time: 4.352, data_time: 3.856, memory: 11791, loss_cls: 0.4358, loss_bbox: 0.3145, loss_segm: 0.0579, loss_mask: 1.2131, loss: 2.0213 2021-05-26 11:54:15,567 - mmdet - INFO - Epoch [8][100/124] lr: 1.000e-03, eta: 5:04:33, time: 4.193, data_time: 3.701, memory: 11791, loss_cls: 0.3799, loss_bbox: 0.2646, loss_segm: 0.0613, loss_mask: 1.0739, loss: 1.7798 2021-05-26 11:56:46,680 - mmdet - INFO - Evaluating bbox... 2021-05-26 11:56:46,833 - mmdet - INFO - Evaluating segm... 2021-05-26 11:56:47,374 - mmdet - INFO - Saving checkpoint at 8 epochs 2021-05-26 11:56:48,055 - mmdet - INFO - Exp name: yolact.py 2021-05-26 11:56:48,055 - mmdet - INFO - Epoch(val) [8][206] bbox_mAP: 0.6060, bbox_mAP_50: 0.8380, bbox_mAP_75: 0.7180, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.4000, bbox_mAP_l: 0.6470, bbox_mAP_copypaste: 0.606 0.838 0.718 -1.000 0.400 0.647, segm_mAP: 0.4870, segm_mAP_50: 0.7860, segm_mAP_75: 0.4650, segm_mAP_s: -1.0000, segm_mAP_m: 0.1000, segm_mAP_l: 0.5430, segm_mAP_copypaste: 0.487 0.786 0.465 -1.000 0.100 0.543 2021-05-26 12:00:20,774 - mmdet - INFO - Epoch [9][50/124] lr: 1.000e-03, eta: 4:56:28, time: 4.252, data_time: 3.745, memory: 11791, loss_cls: 0.3391, loss_bbox: 0.2340, loss_segm: 0.0538, loss_mask: 1.0948, loss: 1.7217 2021-05-26 12:03:53,252 - mmdet - INFO - Epoch [9][100/124] lr: 1.000e-03, eta: 4:56:43, time: 4.250, data_time: 3.750, memory: 11791, loss_cls: 0.4147, loss_bbox: 0.3034, loss_segm: 0.0610, loss_mask: 1.1462, loss: 1.9253 2021-05-26 12:06:40,582 - mmdet - INFO - Evaluating bbox... 2021-05-26 12:06:40,720 - mmdet - INFO - Evaluating segm... 2021-05-26 12:06:40,909 - mmdet - INFO - Saving checkpoint at 9 epochs 2021-05-26 12:06:41,568 - mmdet - INFO - Exp name: yolact.py 2021-05-26 12:06:41,569 - mmdet - INFO - Epoch(val) [9][206] bbox_mAP: 0.5910, bbox_mAP_50: 0.8170, bbox_mAP_75: 0.7010, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3990, bbox_mAP_l: 0.6290, bbox_mAP_copypaste: 0.591 0.817 0.701 -1.000 0.399 0.629, segm_mAP: 0.4640, segm_mAP_50: 0.7800, segm_mAP_75: 0.4400, segm_mAP_s: -1.0000, segm_mAP_m: 0.1380, segm_mAP_l: 0.5130, segm_mAP_copypaste: 0.464 0.780 0.440 -1.000 0.138 0.513 2021-05-26 12:10:02,295 - mmdet - INFO - Epoch [10][50/124] lr: 1.000e-03, eta: 4:48:17, time: 4.013, data_time: 3.521, memory: 11791, loss_cls: 0.3931, loss_bbox: 0.2813, loss_segm: 0.0656, loss_mask: 1.1096, loss: 1.8495 2021-05-26 12:13:23,885 - mmdet - INFO - Epoch [10][100/124] lr: 1.000e-03, eta: 4:47:28, time: 4.032, data_time: 3.532, memory: 11791, loss_cls: 0.3465, loss_bbox: 0.2407, loss_segm: 0.0530, loss_mask: 1.0387, loss: 1.6789 2021-05-26 12:15:59,413 - mmdet - INFO - Evaluating bbox... 2021-05-26 12:15:59,557 - mmdet - INFO - Evaluating segm... 2021-05-26 12:15:59,739 - mmdet - INFO - Saving checkpoint at 10 epochs 2021-05-26 12:16:00,383 - mmdet - INFO - Exp name: yolact.py 2021-05-26 12:16:00,383 - mmdet - INFO - Epoch(val) [10][206] bbox_mAP: 0.5910, bbox_mAP_50: 0.8120, bbox_mAP_75: 0.6910, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3080, bbox_mAP_l: 0.6390, bbox_mAP_copypaste: 0.591 0.812 0.691 -1.000 0.308 0.639, segm_mAP: 0.4610, segm_mAP_50: 0.7570, segm_mAP_75: 0.4320, segm_mAP_s: -1.0000, segm_mAP_m: 0.0990, segm_mAP_l: 0.5120, segm_mAP_copypaste: 0.461 0.757 0.432 -1.000 0.099 0.512 2021-05-26 12:19:48,990 - mmdet - INFO - Epoch [11][50/124] lr: 1.000e-03, eta: 4:41:27, time: 4.570, data_time: 4.078, memory: 11791, loss_cls: 0.3538, loss_bbox: 0.2350, loss_segm: 0.0500, loss_mask: 1.0783, loss: 1.7170 2021-05-26 12:23:18,130 - mmdet - INFO - Epoch [11][100/124] lr: 1.000e-03, eta: 4:40:49, time: 4.183, data_time: 3.696, memory: 11791, loss_cls: 0.3429, loss_bbox: 0.2116, loss_segm: 0.0598, loss_mask: 1.1136, loss: 1.7278 2021-05-26 12:26:05,891 - mmdet - INFO - Evaluating bbox... 2021-05-26 12:26:06,036 - mmdet - INFO - Evaluating segm... 2021-05-26 12:26:06,240 - mmdet - INFO - Saving checkpoint at 11 epochs 2021-05-26 12:26:06,892 - mmdet - INFO - Exp name: yolact.py 2021-05-26 12:26:06,893 - mmdet - INFO - Epoch(val) [11][206] bbox_mAP: 0.5860, bbox_mAP_50: 0.7990, bbox_mAP_75: 0.6940, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3730, bbox_mAP_l: 0.6270, bbox_mAP_copypaste: 0.586 0.799 0.694 -1.000 0.373 0.627, segm_mAP: 0.4290, segm_mAP_50: 0.7700, segm_mAP_75: 0.3720, segm_mAP_s: -1.0000, segm_mAP_m: 0.1310, segm_mAP_l: 0.4750, segm_mAP_copypaste: 0.429 0.770 0.372 -1.000 0.131 0.475 2021-05-26 12:29:23,438 - mmdet - INFO - Epoch [12][50/124] lr: 1.000e-03, eta: 4:33:09, time: 3.929, data_time: 3.426, memory: 11791, loss_cls: 0.3468, loss_bbox: 0.2429, loss_segm: 0.0575, loss_mask: 1.0500, loss: 1.6972 2021-05-26 12:32:50,931 - mmdet - INFO - Epoch [12][100/124] lr: 1.000e-03, eta: 4:32:15, time: 4.150, data_time: 3.654, memory: 11791, loss_cls: 0.3362, loss_bbox: 0.2332, loss_segm: 0.0463, loss_mask: 1.0931, loss: 1.7088 2021-05-26 12:35:31,977 - mmdet - INFO - Evaluating bbox... 2021-05-26 12:35:32,112 - mmdet - INFO - Evaluating segm... 2021-05-26 12:35:32,309 - mmdet - INFO - Saving checkpoint at 12 epochs 2021-05-26 12:35:32,941 - mmdet - INFO - Exp name: yolact.py 2021-05-26 12:35:32,942 - mmdet - INFO - Epoch(val) [12][206] bbox_mAP: 0.5230, bbox_mAP_50: 0.7730, bbox_mAP_75: 0.6220, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3820, bbox_mAP_l: 0.5520, bbox_mAP_copypaste: 0.523 0.773 0.622 -1.000 0.382 0.552, segm_mAP: 0.4380, segm_mAP_50: 0.7320, segm_mAP_75: 0.3990, segm_mAP_s: -1.0000, segm_mAP_m: 0.1460, segm_mAP_l: 0.4830, segm_mAP_copypaste: 0.438 0.732 0.399 -1.000 0.146 0.483 2021-05-26 12:39:04,006 - mmdet - INFO - Epoch [13][50/124] lr: 1.000e-03, eta: 4:25:46, time: 4.219, data_time: 3.729, memory: 11791, loss_cls: 0.2930, loss_bbox: 0.2179, loss_segm: 0.0510, loss_mask: 1.0409, loss: 1.6028 2021-05-26 12:42:26,867 - mmdet - INFO - Epoch [13][100/124] lr: 1.000e-03, eta: 4:24:27, time: 4.057, data_time: 3.555, memory: 11791, loss_cls: 0.3305, loss_bbox: 0.2342, loss_segm: 0.0569, loss_mask: 0.9758, loss: 1.5974 2021-05-26 12:45:06,529 - mmdet - INFO - Evaluating bbox... 2021-05-26 12:45:06,655 - mmdet - INFO - Evaluating segm... 2021-05-26 12:45:06,854 - mmdet - INFO - Saving checkpoint at 13 epochs 2021-05-26 12:45:07,501 - mmdet - INFO - Exp name: yolact.py 2021-05-26 12:45:07,501 - mmdet - INFO - Epoch(val) [13][206] bbox_mAP: 0.5760, bbox_mAP_50: 0.7890, bbox_mAP_75: 0.6650, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3600, bbox_mAP_l: 0.6160, bbox_mAP_copypaste: 0.576 0.789 0.665 -1.000 0.360 0.616, segm_mAP: 0.4630, segm_mAP_50: 0.7310, segm_mAP_75: 0.4720, segm_mAP_s: -1.0000, segm_mAP_m: 0.0920, segm_mAP_l: 0.5160, segm_mAP_copypaste: 0.463 0.731 0.472 -1.000 0.092 0.516 2021-05-26 12:48:47,808 - mmdet - INFO - Epoch [14][50/124] lr: 1.000e-03, eta: 4:18:39, time: 4.404, data_time: 3.900, memory: 11791, loss_cls: 0.3195, loss_bbox: 0.2118, loss_segm: 0.0562, loss_mask: 1.0212, loss: 1.6088 2021-05-26 12:52:28,340 - mmdet - INFO - Epoch [14][100/124] lr: 1.000e-03, eta: 4:17:58, time: 4.411, data_time: 3.923, memory: 11791, loss_cls: 0.3130, loss_bbox: 0.2159, loss_segm: 0.0472, loss_mask: 1.0707, loss: 1.6468 2021-05-26 12:55:05,306 - mmdet - INFO - Evaluating bbox... 2021-05-26 12:55:05,445 - mmdet - INFO - Evaluating segm... 2021-05-26 12:55:05,625 - mmdet - INFO - Saving checkpoint at 14 epochs 2021-05-26 12:55:06,293 - mmdet - INFO - Exp name: yolact.py 2021-05-26 12:55:06,294 - mmdet - INFO - Epoch(val) [14][206] bbox_mAP: 0.6340, bbox_mAP_50: 0.8430, bbox_mAP_75: 0.7320, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3460, bbox_mAP_l: 0.6860, bbox_mAP_copypaste: 0.634 0.843 0.732 -1.000 0.346 0.686, segm_mAP: 0.4830, segm_mAP_50: 0.7940, segm_mAP_75: 0.4650, segm_mAP_s: -1.0000, segm_mAP_m: 0.0970, segm_mAP_l: 0.5390, segm_mAP_copypaste: 0.483 0.794 0.465 -1.000 0.097 0.539 2021-05-26 12:58:29,356 - mmdet - INFO - Epoch [15][50/124] lr: 1.000e-03, eta: 4:11:33, time: 4.059, data_time: 3.549, memory: 11791, loss_cls: 0.3141, loss_bbox: 0.2029, loss_segm: 0.0571, loss_mask: 0.9764, loss: 1.5504 2021-05-26 13:02:14,439 - mmdet - INFO - Epoch [15][100/124] lr: 1.000e-03, eta: 4:10:51, time: 4.502, data_time: 3.997, memory: 11791, loss_cls: 0.2699, loss_bbox: 0.1987, loss_segm: 0.0480, loss_mask: 1.0068, loss: 1.5234 2021-05-26 13:04:51,298 - mmdet - INFO - Evaluating bbox... 2021-05-26 13:04:51,427 - mmdet - INFO - Evaluating segm... 2021-05-26 13:04:51,976 - mmdet - INFO - Saving checkpoint at 15 epochs 2021-05-26 13:04:52,664 - mmdet - INFO - Exp name: yolact.py 2021-05-26 13:04:52,664 - mmdet - INFO - Epoch(val) [15][206] bbox_mAP: 0.6130, bbox_mAP_50: 0.8650, bbox_mAP_75: 0.7170, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.2560, bbox_mAP_l: 0.6680, bbox_mAP_copypaste: 0.613 0.865 0.717 -1.000 0.256 0.668, segm_mAP: 0.4840, segm_mAP_50: 0.8300, segm_mAP_75: 0.4380, segm_mAP_s: -1.0000, segm_mAP_m: 0.0830, segm_mAP_l: 0.5430, segm_mAP_copypaste: 0.484 0.830 0.438 -1.000 0.083 0.543 2021-05-26 13:08:17,363 - mmdet - INFO - Epoch [16][50/124] lr: 1.000e-03, eta: 4:04:42, time: 4.092, data_time: 3.588, memory: 11791, loss_cls: 0.3398, loss_bbox: 0.2091, loss_segm: 0.0563, loss_mask: 1.0166, loss: 1.6218 2021-05-26 13:11:45,643 - mmdet - INFO - Epoch [16][100/124] lr: 1.000e-03, eta: 4:03:11, time: 4.165, data_time: 3.668, memory: 11791, loss_cls: 0.3014, loss_bbox: 0.1873, loss_segm: 0.0573, loss_mask: 1.0053, loss: 1.5513 2021-05-26 13:14:25,091 - mmdet - INFO - Evaluating bbox... 2021-05-26 13:14:25,217 - mmdet - INFO - Evaluating segm... 2021-05-26 13:14:25,414 - mmdet - INFO - Saving checkpoint at 16 epochs 2021-05-26 13:14:26,089 - mmdet - INFO - Exp name: yolact.py 2021-05-26 13:14:26,090 - mmdet - INFO - Epoch(val) [16][206] bbox_mAP: 0.5860, bbox_mAP_50: 0.8260, bbox_mAP_75: 0.6540, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3220, bbox_mAP_l: 0.6320, bbox_mAP_copypaste: 0.586 0.826 0.654 -1.000 0.322 0.632, segm_mAP: 0.4860, segm_mAP_50: 0.8030, segm_mAP_75: 0.4540, segm_mAP_s: -1.0000, segm_mAP_m: 0.1390, segm_mAP_l: 0.5380, segm_mAP_copypaste: 0.486 0.803 0.454 -1.000 0.139 0.538 2021-05-26 13:17:48,784 - mmdet - INFO - Epoch [17][50/124] lr: 1.000e-03, eta: 3:57:10, time: 4.052, data_time: 3.558, memory: 11791, loss_cls: 0.3046, loss_bbox: 0.1775, loss_segm: 0.0478, loss_mask: 0.9899, loss: 1.5198 2021-05-26 13:20:49,591 - mmdet - INFO - Epoch [17][100/124] lr: 1.000e-03, eta: 3:54:39, time: 3.616, data_time: 3.121, memory: 11791, loss_cls: 0.2753, loss_bbox: 0.1775, loss_segm: 0.0494, loss_mask: 0.9872, loss: 1.4893 2021-05-26 13:23:31,337 - mmdet - INFO - Evaluating bbox... 2021-05-26 13:23:31,477 - mmdet - INFO - Evaluating segm... 2021-05-26 13:23:31,659 - mmdet - INFO - Saving checkpoint at 17 epochs 2021-05-26 13:23:32,327 - mmdet - INFO - Exp name: yolact.py 2021-05-26 13:23:32,327 - mmdet - INFO - Epoch(val) [17][206] bbox_mAP: 0.6120, bbox_mAP_50: 0.8190, bbox_mAP_75: 0.7040, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3960, bbox_mAP_l: 0.6550, bbox_mAP_copypaste: 0.612 0.819 0.704 -1.000 0.396 0.655, segm_mAP: 0.4880, segm_mAP_50: 0.7890, segm_mAP_75: 0.4910, segm_mAP_s: -1.0000, segm_mAP_m: 0.1400, segm_mAP_l: 0.5400, segm_mAP_copypaste: 0.488 0.789 0.491 -1.000 0.140 0.540 2021-05-26 13:27:14,660 - mmdet - INFO - Epoch [18][50/124] lr: 1.000e-03, eta: 3:49:28, time: 4.445, data_time: 3.934, memory: 11791, loss_cls: 0.3365, loss_bbox: 0.2021, loss_segm: 0.0517, loss_mask: 1.0227, loss: 1.6129 2021-05-26 13:30:45,041 - mmdet - INFO - Epoch [18][100/124] lr: 1.000e-03, eta: 3:47:50, time: 4.208, data_time: 3.717, memory: 11791, loss_cls: 0.2897, loss_bbox: 0.1853, loss_segm: 0.0457, loss_mask: 1.0002, loss: 1.5209 2021-05-26 13:33:34,985 - mmdet - INFO - Evaluating bbox... 2021-05-26 13:33:35,124 - mmdet - INFO - Evaluating segm... 2021-05-26 13:33:35,306 - mmdet - INFO - Saving checkpoint at 18 epochs 2021-05-26 13:33:35,978 - mmdet - INFO - Exp name: yolact.py 2021-05-26 13:33:35,978 - mmdet - INFO - Epoch(val) [18][206] bbox_mAP: 0.6190, bbox_mAP_50: 0.8460, bbox_mAP_75: 0.6870, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3130, bbox_mAP_l: 0.6720, bbox_mAP_copypaste: 0.619 0.846 0.687 -1.000 0.313 0.672, segm_mAP: 0.5080, segm_mAP_50: 0.8240, segm_mAP_75: 0.5060, segm_mAP_s: -1.0000, segm_mAP_m: 0.1390, segm_mAP_l: 0.5620, segm_mAP_copypaste: 0.508 0.824 0.506 -1.000 0.139 0.562 2021-05-26 13:37:04,894 - mmdet - INFO - Epoch [19][50/124] lr: 1.000e-03, eta: 3:42:20, time: 4.176, data_time: 3.671, memory: 11791, loss_cls: 0.2885, loss_bbox: 0.1912, loss_segm: 0.0478, loss_mask: 0.9572, loss: 1.4847 2021-05-26 13:40:32,286 - mmdet - INFO - Epoch [19][100/124] lr: 1.000e-03, eta: 3:40:31, time: 4.148, data_time: 3.660, memory: 11791, loss_cls: 0.2860, loss_bbox: 0.1964, loss_segm: 0.0495, loss_mask: 0.9860, loss: 1.5179 2021-05-26 13:43:12,675 - mmdet - INFO - Evaluating bbox... 2021-05-26 13:43:12,815 - mmdet - INFO - Evaluating segm... 2021-05-26 13:43:13,011 - mmdet - INFO - Saving checkpoint at 19 epochs 2021-05-26 13:43:13,657 - mmdet - INFO - Exp name: yolact.py 2021-05-26 13:43:13,657 - mmdet - INFO - Epoch(val) [19][206] bbox_mAP: 0.5950, bbox_mAP_50: 0.8140, bbox_mAP_75: 0.6800, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.2480, bbox_mAP_l: 0.6490, bbox_mAP_copypaste: 0.595 0.814 0.680 -1.000 0.248 0.649, segm_mAP: 0.4480, segm_mAP_50: 0.7860, segm_mAP_75: 0.3970, segm_mAP_s: -1.0000, segm_mAP_m: 0.1030, segm_mAP_l: 0.4970, segm_mAP_copypaste: 0.448 0.786 0.397 -1.000 0.103 0.497 2021-05-26 13:46:48,763 - mmdet - INFO - Epoch [20][50/124] lr: 1.000e-03, eta: 3:35:18, time: 4.300, data_time: 3.803, memory: 11791, loss_cls: 0.3014, loss_bbox: 0.1904, loss_segm: 0.0488, loss_mask: 0.9915, loss: 1.5322 2021-05-26 13:50:06,727 - mmdet - INFO - Epoch [20][100/124] lr: 1.000e-03, eta: 3:33:10, time: 3.959, data_time: 3.453, memory: 11791, loss_cls: 0.3202, loss_bbox: 0.2165, loss_segm: 0.0568, loss_mask: 0.9928, loss: 1.5863 2021-05-26 13:52:52,224 - mmdet - INFO - Evaluating bbox... 2021-05-26 13:52:52,356 - mmdet - INFO - Evaluating segm... 2021-05-26 13:52:52,554 - mmdet - INFO - Saving checkpoint at 20 epochs 2021-05-26 13:52:53,240 - mmdet - INFO - Exp name: yolact.py 2021-05-26 13:52:53,241 - mmdet - INFO - Epoch(val) [20][206] bbox_mAP: 0.5940, bbox_mAP_50: 0.8080, bbox_mAP_75: 0.6740, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3030, bbox_mAP_l: 0.6430, bbox_mAP_copypaste: 0.594 0.808 0.674 -1.000 0.303 0.643, segm_mAP: 0.4920, segm_mAP_50: 0.7920, segm_mAP_75: 0.4950, segm_mAP_s: -1.0000, segm_mAP_m: 0.1370, segm_mAP_l: 0.5440, segm_mAP_copypaste: 0.492 0.792 0.495 -1.000 0.137 0.544 2021-05-26 13:56:23,225 - mmdet - INFO - Epoch [21][50/124] lr: 1.000e-04, eta: 3:27:55, time: 4.198, data_time: 3.704, memory: 11791, loss_cls: 0.2883, loss_bbox: 0.1868, loss_segm: 0.0526, loss_mask: 1.0077, loss: 1.5354 2021-05-26 14:00:15,335 - mmdet - INFO - Epoch [21][100/124] lr: 1.000e-04, eta: 3:26:32, time: 4.642, data_time: 4.158, memory: 11791, loss_cls: 0.2701, loss_bbox: 0.1560, loss_segm: 0.0476, loss_mask: 0.9604, loss: 1.4342 2021-05-26 14:03:00,297 - mmdet - INFO - Evaluating bbox... 2021-05-26 14:03:00,443 - mmdet - INFO - Evaluating segm... 2021-05-26 14:03:01,001 - mmdet - INFO - Saving checkpoint at 21 epochs 2021-05-26 14:03:01,698 - mmdet - INFO - Exp name: yolact.py 2021-05-26 14:03:01,698 - mmdet - INFO - Epoch(val) [21][206] bbox_mAP: 0.6110, bbox_mAP_50: 0.8360, bbox_mAP_75: 0.6990, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.2890, bbox_mAP_l: 0.6610, bbox_mAP_copypaste: 0.611 0.836 0.699 -1.000 0.289 0.661, segm_mAP: 0.5050, segm_mAP_50: 0.8070, segm_mAP_75: 0.4990, segm_mAP_s: -1.0000, segm_mAP_m: 0.1230, segm_mAP_l: 0.5620, segm_mAP_copypaste: 0.505 0.807 0.499 -1.000 0.123 0.562 2021-05-26 14:06:31,599 - mmdet - INFO - Epoch [22][50/124] lr: 1.000e-04, eta: 3:21:21, time: 4.196, data_time: 3.700, memory: 11791, loss_cls: 0.2428, loss_bbox: 0.1434, loss_segm: 0.0409, loss_mask: 0.8945, loss: 1.3216 2021-05-26 14:10:17,179 - mmdet - INFO - Epoch [22][100/124] lr: 1.000e-04, eta: 3:19:42, time: 4.512, data_time: 4.023, memory: 11791, loss_cls: 0.2828, loss_bbox: 0.1808, loss_segm: 0.0476, loss_mask: 0.9159, loss: 1.4270 2021-05-26 14:13:00,381 - mmdet - INFO - Evaluating bbox... 2021-05-26 14:13:00,540 - mmdet - INFO - Evaluating segm... 2021-05-26 14:13:00,747 - mmdet - INFO - Saving checkpoint at 22 epochs 2021-05-26 14:13:01,433 - mmdet - INFO - Exp name: yolact.py 2021-05-26 14:13:01,434 - mmdet - INFO - Epoch(val) [22][206] bbox_mAP: 0.6200, bbox_mAP_50: 0.8340, bbox_mAP_75: 0.6970, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3050, bbox_mAP_l: 0.6720, bbox_mAP_copypaste: 0.620 0.834 0.697 -1.000 0.305 0.672, segm_mAP: 0.5000, segm_mAP_50: 0.8040, segm_mAP_75: 0.4860, segm_mAP_s: -1.0000, segm_mAP_m: 0.1180, segm_mAP_l: 0.5570, segm_mAP_copypaste: 0.500 0.804 0.486 -1.000 0.118 0.557 2021-05-26 14:16:39,344 - mmdet - INFO - Epoch [23][50/124] lr: 1.000e-04, eta: 3:14:44, time: 4.356, data_time: 3.859, memory: 11791, loss_cls: 0.2543, loss_bbox: 0.1551, loss_segm: 0.0477, loss_mask: 0.9337, loss: 1.3908 2021-05-26 14:20:07,777 - mmdet - INFO - Epoch [23][100/124] lr: 1.000e-04, eta: 3:12:38, time: 4.169, data_time: 3.659, memory: 11791, loss_cls: 0.2295, loss_bbox: 0.1233, loss_segm: 0.0419, loss_mask: 0.8932, loss: 1.2879 2021-05-26 14:22:59,875 - mmdet - INFO - Evaluating bbox... 2021-05-26 14:23:00,005 - mmdet - INFO - Evaluating segm... 2021-05-26 14:23:00,206 - mmdet - INFO - Saving checkpoint at 23 epochs 2021-05-26 14:23:00,911 - mmdet - INFO - Exp name: yolact.py 2021-05-26 14:23:00,912 - mmdet - INFO - Epoch(val) [23][206] bbox_mAP: 0.6130, bbox_mAP_50: 0.8240, bbox_mAP_75: 0.6880, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3070, bbox_mAP_l: 0.6650, bbox_mAP_copypaste: 0.613 0.824 0.688 -1.000 0.307 0.665, segm_mAP: 0.4960, segm_mAP_50: 0.7950, segm_mAP_75: 0.4920, segm_mAP_s: -1.0000, segm_mAP_m: 0.1180, segm_mAP_l: 0.5510, segm_mAP_copypaste: 0.496 0.795 0.492 -1.000 0.118 0.551 2021-05-26 14:26:50,402 - mmdet - INFO - Epoch [24][50/124] lr: 1.000e-04, eta: 3:07:57, time: 4.588, data_time: 4.089, memory: 11791, loss_cls: 0.2415, loss_bbox: 0.1226, loss_segm: 0.0467, loss_mask: 0.9251, loss: 1.3359 2021-05-26 14:30:35,706 - mmdet - INFO - Epoch [24][100/124] lr: 1.000e-04, eta: 3:06:06, time: 4.506, data_time: 4.011, memory: 11791, loss_cls: 0.2697, loss_bbox: 0.1445, loss_segm: 0.0456, loss_mask: 0.9034, loss: 1.3632 2021-05-26 14:33:16,925 - mmdet - INFO - Evaluating bbox... 2021-05-26 14:33:17,065 - mmdet - INFO - Evaluating segm... 2021-05-26 14:33:17,267 - mmdet - INFO - Saving checkpoint at 24 epochs 2021-05-26 14:33:17,907 - mmdet - INFO - Exp name: yolact.py 2021-05-26 14:33:17,907 - mmdet - INFO - Epoch(val) [24][206] bbox_mAP: 0.6210, bbox_mAP_50: 0.8320, bbox_mAP_75: 0.6950, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3010, bbox_mAP_l: 0.6740, bbox_mAP_copypaste: 0.621 0.832 0.695 -1.000 0.301 0.674, segm_mAP: 0.5010, segm_mAP_50: 0.8030, segm_mAP_75: 0.5010, segm_mAP_s: -1.0000, segm_mAP_m: 0.1210, segm_mAP_l: 0.5570, segm_mAP_copypaste: 0.501 0.803 0.501 -1.000 0.121 0.557 2021-05-26 14:36:51,863 - mmdet - INFO - Epoch [25][50/124] lr: 1.000e-04, eta: 3:01:09, time: 4.277, data_time: 3.777, memory: 11791, loss_cls: 0.2635, loss_bbox: 0.1518, loss_segm: 0.0511, loss_mask: 0.9161, loss: 1.3825 2021-05-26 14:40:26,749 - mmdet - INFO - Epoch [25][100/124] lr: 1.000e-04, eta: 2:59:02, time: 4.298, data_time: 3.796, memory: 11791, loss_cls: 0.2148, loss_bbox: 0.1239, loss_segm: 0.0463, loss_mask: 0.8739, loss: 1.2590 2021-05-26 14:43:07,583 - mmdet - INFO - Evaluating bbox... 2021-05-26 14:43:07,726 - mmdet - INFO - Evaluating segm... 2021-05-26 14:43:07,915 - mmdet - INFO - Saving checkpoint at 25 epochs 2021-05-26 14:43:08,588 - mmdet - INFO - Exp name: yolact.py 2021-05-26 14:43:08,588 - mmdet - INFO - Epoch(val) [25][206] bbox_mAP: 0.6140, bbox_mAP_50: 0.8330, bbox_mAP_75: 0.6930, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3110, bbox_mAP_l: 0.6650, bbox_mAP_copypaste: 0.614 0.833 0.693 -1.000 0.311 0.665, segm_mAP: 0.4950, segm_mAP_50: 0.8030, segm_mAP_75: 0.4900, segm_mAP_s: -1.0000, segm_mAP_m: 0.1220, segm_mAP_l: 0.5490, segm_mAP_copypaste: 0.495 0.803 0.490 -1.000 0.122 0.549 2021-05-26 14:46:30,553 - mmdet - INFO - Epoch [26][50/124] lr: 1.000e-04, eta: 2:53:57, time: 4.037, data_time: 3.539, memory: 11791, loss_cls: 0.2629, loss_bbox: 0.1270, loss_segm: 0.0357, loss_mask: 0.8823, loss: 1.3080 2021-05-26 14:49:52,989 - mmdet - INFO - Epoch [26][100/124] lr: 1.000e-04, eta: 2:51:35, time: 4.049, data_time: 3.542, memory: 11791, loss_cls: 0.2727, loss_bbox: 0.1464, loss_segm: 0.0468, loss_mask: 0.9159, loss: 1.3817 2021-05-26 14:52:30,764 - mmdet - INFO - Evaluating bbox... 2021-05-26 14:52:30,907 - mmdet - INFO - Evaluating segm... 2021-05-26 14:52:31,088 - mmdet - INFO - Saving checkpoint at 26 epochs 2021-05-26 14:52:31,748 - mmdet - INFO - Exp name: yolact.py 2021-05-26 14:52:31,749 - mmdet - INFO - Epoch(val) [26][206] bbox_mAP: 0.6290, bbox_mAP_50: 0.8390, bbox_mAP_75: 0.6990, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3190, bbox_mAP_l: 0.6810, bbox_mAP_copypaste: 0.629 0.839 0.699 -1.000 0.319 0.681, segm_mAP: 0.5000, segm_mAP_50: 0.8170, segm_mAP_75: 0.4960, segm_mAP_s: -1.0000, segm_mAP_m: 0.1240, segm_mAP_l: 0.5550, segm_mAP_copypaste: 0.500 0.817 0.496 -1.000 0.124 0.555 2021-05-26 14:56:13,637 - mmdet - INFO - Epoch [27][50/124] lr: 1.000e-04, eta: 2:46:52, time: 4.436, data_time: 3.946, memory: 11791, loss_cls: 0.2430, loss_bbox: 0.1260, loss_segm: 0.0478, loss_mask: 0.8928, loss: 1.3096 2021-05-26 14:59:41,127 - mmdet - INFO - Epoch [27][100/124] lr: 1.000e-04, eta: 2:44:33, time: 4.150, data_time: 3.649, memory: 11791, loss_cls: 0.2743, loss_bbox: 0.1448, loss_segm: 0.0450, loss_mask: 0.8865, loss: 1.3506 2021-05-26 15:02:24,431 - mmdet - INFO - Evaluating bbox... 2021-05-26 15:02:24,554 - mmdet - INFO - Evaluating segm... 2021-05-26 15:02:24,752 - mmdet - INFO - Saving checkpoint at 27 epochs 2021-05-26 15:02:25,440 - mmdet - INFO - Exp name: yolact.py 2021-05-26 15:02:25,441 - mmdet - INFO - Epoch(val) [27][206] bbox_mAP: 0.6300, bbox_mAP_50: 0.8490, bbox_mAP_75: 0.7080, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3100, bbox_mAP_l: 0.6830, bbox_mAP_copypaste: 0.630 0.849 0.708 -1.000 0.310 0.683, segm_mAP: 0.5080, segm_mAP_50: 0.8130, segm_mAP_75: 0.5040, segm_mAP_s: -1.0000, segm_mAP_m: 0.1110, segm_mAP_l: 0.5670, segm_mAP_copypaste: 0.508 0.813 0.504 -1.000 0.111 0.567 2021-05-26 15:05:57,664 - mmdet - INFO - Epoch [28][50/124] lr: 1.000e-04, eta: 2:39:44, time: 4.242, data_time: 3.738, memory: 11791, loss_cls: 0.2729, loss_bbox: 0.1560, loss_segm: 0.0465, loss_mask: 0.9185, loss: 1.3938 2021-05-26 15:09:48,148 - mmdet - INFO - Epoch [28][100/124] lr: 1.000e-04, eta: 2:37:40, time: 4.610, data_time: 4.106, memory: 11791, loss_cls: 0.2491, loss_bbox: 0.1507, loss_segm: 0.0375, loss_mask: 0.9353, loss: 1.3726 2021-05-26 15:12:31,733 - mmdet - INFO - Evaluating bbox... 2021-05-26 15:12:31,885 - mmdet - INFO - Evaluating segm... 2021-05-26 15:12:32,088 - mmdet - INFO - Saving checkpoint at 28 epochs 2021-05-26 15:12:32,779 - mmdet - INFO - Exp name: yolact.py 2021-05-26 15:12:32,779 - mmdet - INFO - Epoch(val) [28][206] bbox_mAP: 0.6200, bbox_mAP_50: 0.8340, bbox_mAP_75: 0.6940, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3010, bbox_mAP_l: 0.6730, bbox_mAP_copypaste: 0.620 0.834 0.694 -1.000 0.301 0.673, segm_mAP: 0.4930, segm_mAP_50: 0.7970, segm_mAP_75: 0.4880, segm_mAP_s: -1.0000, segm_mAP_m: 0.1070, segm_mAP_l: 0.5490, segm_mAP_copypaste: 0.493 0.797 0.488 -1.000 0.107 0.549 2021-05-26 15:15:43,488 - mmdet - INFO - Epoch [29][50/124] lr: 1.000e-04, eta: 2:32:38, time: 3.812, data_time: 3.316, memory: 11791, loss_cls: 0.2448, loss_bbox: 0.1238, loss_segm: 0.0446, loss_mask: 0.8435, loss: 1.2568 2021-05-26 15:19:50,889 - mmdet - INFO - Epoch [29][100/124] lr: 1.000e-04, eta: 2:30:43, time: 4.947, data_time: 4.429, memory: 11791, loss_cls: 0.2654, loss_bbox: 0.1642, loss_segm: 0.0493, loss_mask: 0.8980, loss: 1.3769 2021-05-26 15:23:04,966 - mmdet - INFO - Evaluating bbox... 2021-05-26 15:23:05,121 - mmdet - INFO - Evaluating segm... 2021-05-26 15:23:05,336 - mmdet - INFO - Saving checkpoint at 29 epochs 2021-05-26 15:23:06,060 - mmdet - INFO - Exp name: yolact.py 2021-05-26 15:23:06,060 - mmdet - INFO - Epoch(val) [29][206] bbox_mAP: 0.6250, bbox_mAP_50: 0.8400, bbox_mAP_75: 0.7000, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3130, bbox_mAP_l: 0.6780, bbox_mAP_copypaste: 0.625 0.840 0.700 -1.000 0.313 0.678, segm_mAP: 0.5010, segm_mAP_50: 0.8040, segm_mAP_75: 0.4980, segm_mAP_s: -1.0000, segm_mAP_m: 0.1120, segm_mAP_l: 0.5580, segm_mAP_copypaste: 0.501 0.804 0.498 -1.000 0.112 0.558 2021-05-26 15:26:46,749 - mmdet - INFO - Epoch [30][50/124] lr: 1.000e-04, eta: 2:26:04, time: 4.412, data_time: 3.912, memory: 11791, loss_cls: 0.2904, loss_bbox: 0.1512, loss_segm: 0.0415, loss_mask: 0.9113, loss: 1.3943 2021-05-26 15:30:26,692 - mmdet - INFO - Epoch [30][100/124] lr: 1.000e-04, eta: 2:23:45, time: 4.399, data_time: 3.899, memory: 11791, loss_cls: 0.2488, loss_bbox: 0.1259, loss_segm: 0.0479, loss_mask: 0.9004, loss: 1.3229 2021-05-26 15:33:16,085 - mmdet - INFO - Evaluating bbox... 2021-05-26 15:33:16,281 - mmdet - INFO - Evaluating segm... 2021-05-26 15:33:16,470 - mmdet - INFO - Saving checkpoint at 30 epochs 2021-05-26 15:33:17,198 - mmdet - INFO - Exp name: yolact.py 2021-05-26 15:33:17,198 - mmdet - INFO - Epoch(val) [30][206] bbox_mAP: 0.6260, bbox_mAP_50: 0.8430, bbox_mAP_75: 0.7050, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3190, bbox_mAP_l: 0.6770, bbox_mAP_copypaste: 0.626 0.843 0.705 -1.000 0.319 0.677, segm_mAP: 0.4970, segm_mAP_50: 0.8070, segm_mAP_75: 0.4960, segm_mAP_s: -1.0000, segm_mAP_m: 0.0990, segm_mAP_l: 0.5550, segm_mAP_copypaste: 0.497 0.807 0.496 -1.000 0.099 0.555 2021-05-26 15:37:07,117 - mmdet - INFO - Epoch [31][50/124] lr: 1.000e-04, eta: 2:19:14, time: 4.596, data_time: 4.096, memory: 11791, loss_cls: 0.2913, loss_bbox: 0.1519, loss_segm: 0.0422, loss_mask: 0.9069, loss: 1.3922 2021-05-26 15:40:28,611 - mmdet - INFO - Epoch [31][100/124] lr: 1.000e-04, eta: 2:16:41, time: 4.030, data_time: 3.528, memory: 11791, loss_cls: 0.2034, loss_bbox: 0.1343, loss_segm: 0.0457, loss_mask: 0.8722, loss: 1.2556 2021-05-26 15:43:06,450 - mmdet - INFO - Evaluating bbox... 2021-05-26 15:43:06,609 - mmdet - INFO - Evaluating segm... 2021-05-26 15:43:06,826 - mmdet - INFO - Saving checkpoint at 31 epochs 2021-05-26 15:43:08,170 - mmdet - INFO - Exp name: yolact.py 2021-05-26 15:43:08,170 - mmdet - INFO - Epoch(val) [31][206] bbox_mAP: 0.6230, bbox_mAP_50: 0.8350, bbox_mAP_75: 0.6880, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3200, bbox_mAP_l: 0.6750, bbox_mAP_copypaste: 0.623 0.835 0.688 -1.000 0.320 0.675, segm_mAP: 0.4910, segm_mAP_50: 0.7990, segm_mAP_75: 0.4860, segm_mAP_s: -1.0000, segm_mAP_m: 0.1040, segm_mAP_l: 0.5460, segm_mAP_copypaste: 0.491 0.799 0.486 -1.000 0.104 0.546 2021-05-26 15:46:41,426 - mmdet - INFO - Epoch [32][50/124] lr: 1.000e-04, eta: 2:12:01, time: 4.263, data_time: 3.773, memory: 11791, loss_cls: 0.2671, loss_bbox: 0.1439, loss_segm: 0.0465, loss_mask: 0.9158, loss: 1.3734 2021-05-26 15:50:05,607 - mmdet - INFO - Epoch [32][100/124] lr: 1.000e-04, eta: 2:09:28, time: 4.084, data_time: 3.585, memory: 11791, loss_cls: 0.2456, loss_bbox: 0.1473, loss_segm: 0.0411, loss_mask: 0.8980, loss: 1.3320 2021-05-26 15:52:42,127 - mmdet - INFO - Evaluating bbox... 2021-05-26 15:52:42,274 - mmdet - INFO - Evaluating segm... 2021-05-26 15:52:42,458 - mmdet - INFO - Saving checkpoint at 32 epochs 2021-05-26 15:52:43,156 - mmdet - INFO - Exp name: yolact.py 2021-05-26 15:52:43,156 - mmdet - INFO - Epoch(val) [32][206] bbox_mAP: 0.6220, bbox_mAP_50: 0.8340, bbox_mAP_75: 0.6930, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.2960, bbox_mAP_l: 0.6760, bbox_mAP_copypaste: 0.622 0.834 0.693 -1.000 0.296 0.676, segm_mAP: 0.4930, segm_mAP_50: 0.7990, segm_mAP_75: 0.4900, segm_mAP_s: -1.0000, segm_mAP_m: 0.1020, segm_mAP_l: 0.5500, segm_mAP_copypaste: 0.493 0.799 0.490 -1.000 0.102 0.550 2021-05-26 15:56:15,535 - mmdet - INFO - Epoch [33][50/124] lr: 1.000e-04, eta: 2:04:50, time: 4.246, data_time: 3.754, memory: 11791, loss_cls: 0.2008, loss_bbox: 0.1262, loss_segm: 0.0514, loss_mask: 0.8551, loss: 1.2336 2021-05-26 15:59:37,488 - mmdet - INFO - Epoch [33][100/124] lr: 1.000e-04, eta: 2:02:14, time: 4.039, data_time: 3.556, memory: 11791, loss_cls: 0.2436, loss_bbox: 0.1468, loss_segm: 0.0413, loss_mask: 0.8807, loss: 1.3124 2021-05-26 16:02:26,064 - mmdet - INFO - Evaluating bbox... 2021-05-26 16:02:26,210 - mmdet - INFO - Evaluating segm... 2021-05-26 16:02:26,405 - mmdet - INFO - Saving checkpoint at 33 epochs 2021-05-26 16:02:27,082 - mmdet - INFO - Exp name: yolact.py 2021-05-26 16:02:27,083 - mmdet - INFO - Epoch(val) [33][206] bbox_mAP: 0.6040, bbox_mAP_50: 0.8190, bbox_mAP_75: 0.6770, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3160, bbox_mAP_l: 0.6540, bbox_mAP_copypaste: 0.604 0.819 0.677 -1.000 0.316 0.654, segm_mAP: 0.4770, segm_mAP_50: 0.7820, segm_mAP_75: 0.4730, segm_mAP_s: -1.0000, segm_mAP_m: 0.0990, segm_mAP_l: 0.5310, segm_mAP_copypaste: 0.477 0.782 0.473 -1.000 0.099 0.531 2021-05-26 16:06:08,051 - mmdet - INFO - Epoch [34][50/124] lr: 1.000e-04, eta: 1:57:43, time: 4.417, data_time: 3.915, memory: 11791, loss_cls: 0.2355, loss_bbox: 0.1387, loss_segm: 0.0426, loss_mask: 0.8891, loss: 1.3059 2021-05-26 16:09:47,958 - mmdet - INFO - Epoch [34][100/124] lr: 1.000e-04, eta: 1:55:14, time: 4.398, data_time: 3.906, memory: 11791, loss_cls: 0.2608, loss_bbox: 0.1534, loss_segm: 0.0446, loss_mask: 0.9231, loss: 1.3819 2021-05-26 16:12:26,626 - mmdet - INFO - Evaluating bbox... 2021-05-26 16:12:26,776 - mmdet - INFO - Evaluating segm... 2021-05-26 16:12:26,979 - mmdet - INFO - Saving checkpoint at 34 epochs 2021-05-26 16:12:27,700 - mmdet - INFO - Exp name: yolact.py 2021-05-26 16:12:27,700 - mmdet - INFO - Epoch(val) [34][206] bbox_mAP: 0.6230, bbox_mAP_50: 0.8320, bbox_mAP_75: 0.6910, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3050, bbox_mAP_l: 0.6740, bbox_mAP_copypaste: 0.623 0.832 0.691 -1.000 0.305 0.674, segm_mAP: 0.4960, segm_mAP_50: 0.8020, segm_mAP_75: 0.4900, segm_mAP_s: -1.0000, segm_mAP_m: 0.1150, segm_mAP_l: 0.5520, segm_mAP_copypaste: 0.496 0.802 0.490 -1.000 0.115 0.552 2021-05-26 16:16:08,453 - mmdet - INFO - Epoch [35][50/124] lr: 1.000e-04, eta: 1:50:44, time: 4.413, data_time: 3.921, memory: 11791, loss_cls: 0.2169, loss_bbox: 0.1257, loss_segm: 0.0435, loss_mask: 0.8809, loss: 1.2671 2021-05-26 16:19:52,339 - mmdet - INFO - Epoch [35][100/124] lr: 1.000e-04, eta: 1:48:15, time: 4.478, data_time: 3.976, memory: 11791, loss_cls: 0.2260, loss_bbox: 0.1485, loss_segm: 0.0460, loss_mask: 0.8659, loss: 1.2865 2021-05-26 16:22:36,301 - mmdet - INFO - Evaluating bbox... 2021-05-26 16:22:36,452 - mmdet - INFO - Evaluating segm... 2021-05-26 16:22:36,656 - mmdet - INFO - Saving checkpoint at 35 epochs 2021-05-26 16:22:37,362 - mmdet - INFO - Exp name: yolact.py 2021-05-26 16:22:37,362 - mmdet - INFO - Epoch(val) [35][206] bbox_mAP: 0.6270, bbox_mAP_50: 0.8340, bbox_mAP_75: 0.6960, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3100, bbox_mAP_l: 0.6790, bbox_mAP_copypaste: 0.627 0.834 0.696 -1.000 0.310 0.679, segm_mAP: 0.4950, segm_mAP_50: 0.7980, segm_mAP_75: 0.4910, segm_mAP_s: -1.0000, segm_mAP_m: 0.1100, segm_mAP_l: 0.5510, segm_mAP_copypaste: 0.495 0.798 0.491 -1.000 0.110 0.551 2021-05-26 16:25:46,107 - mmdet - INFO - Epoch [36][50/124] lr: 1.000e-04, eta: 1:43:32, time: 3.773, data_time: 3.279, memory: 11791, loss_cls: 0.2050, loss_bbox: 0.1225, loss_segm: 0.0469, loss_mask: 0.8961, loss: 1.2705 2021-05-26 16:29:24,164 - mmdet - INFO - Epoch [36][100/124] lr: 1.000e-04, eta: 1:40:59, time: 4.361, data_time: 3.854, memory: 11791, loss_cls: 0.2495, loss_bbox: 0.1424, loss_segm: 0.0436, loss_mask: 0.8518, loss: 1.2873 2021-05-26 16:32:17,167 - mmdet - INFO - Evaluating bbox... 2021-05-26 16:32:17,696 - mmdet - INFO - Evaluating segm... 2021-05-26 16:32:17,889 - mmdet - INFO - Saving checkpoint at 36 epochs 2021-05-26 16:32:18,598 - mmdet - INFO - Exp name: yolact.py 2021-05-26 16:32:18,598 - mmdet - INFO - Epoch(val) [36][206] bbox_mAP: 0.6100, bbox_mAP_50: 0.8200, bbox_mAP_75: 0.6840, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3050, bbox_mAP_l: 0.6610, bbox_mAP_copypaste: 0.610 0.820 0.684 -1.000 0.305 0.661, segm_mAP: 0.4820, segm_mAP_50: 0.7780, segm_mAP_75: 0.4800, segm_mAP_s: -1.0000, segm_mAP_m: 0.0970, segm_mAP_l: 0.5380, segm_mAP_copypaste: 0.482 0.778 0.480 -1.000 0.097 0.538 2021-05-26 16:35:54,805 - mmdet - INFO - Epoch [37][50/124] lr: 1.000e-04, eta: 1:36:30, time: 4.320, data_time: 3.820, memory: 11791, loss_cls: 0.2338, loss_bbox: 0.1404, loss_segm: 0.0429, loss_mask: 0.8961, loss: 1.3131 2021-05-26 16:39:30,225 - mmdet - INFO - Epoch [37][100/124] lr: 1.000e-04, eta: 1:33:54, time: 4.308, data_time: 3.801, memory: 11791, loss_cls: 0.2218, loss_bbox: 0.1277, loss_segm: 0.0412, loss_mask: 0.8544, loss: 1.2450 2021-05-26 16:42:19,250 - mmdet - INFO - Evaluating bbox... 2021-05-26 16:42:19,399 - mmdet - INFO - Evaluating segm... 2021-05-26 16:42:19,590 - mmdet - INFO - Saving checkpoint at 37 epochs 2021-05-26 16:42:20,307 - mmdet - INFO - Exp name: yolact.py 2021-05-26 16:42:20,307 - mmdet - INFO - Epoch(val) [37][206] bbox_mAP: 0.6240, bbox_mAP_50: 0.8330, bbox_mAP_75: 0.6980, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3400, bbox_mAP_l: 0.6750, bbox_mAP_copypaste: 0.624 0.833 0.698 -1.000 0.340 0.675, segm_mAP: 0.4960, segm_mAP_50: 0.8030, segm_mAP_75: 0.4930, segm_mAP_s: -1.0000, segm_mAP_m: 0.1180, segm_mAP_l: 0.5510, segm_mAP_copypaste: 0.496 0.803 0.493 -1.000 0.118 0.551 2021-05-26 16:45:45,687 - mmdet - INFO - Epoch [38][50/124] lr: 1.000e-04, eta: 1:29:22, time: 4.106, data_time: 3.613, memory: 11791, loss_cls: 0.2428, loss_bbox: 0.1425, loss_segm: 0.0440, loss_mask: 0.9193, loss: 1.3487 2021-05-26 16:49:18,918 - mmdet - INFO - Epoch [38][100/124] lr: 1.000e-04, eta: 1:26:44, time: 4.265, data_time: 3.764, memory: 11791, loss_cls: 0.2191, loss_bbox: 0.1376, loss_segm: 0.0428, loss_mask: 0.8540, loss: 1.2535 2021-05-26 16:52:11,233 - mmdet - INFO - Evaluating bbox... 2021-05-26 16:52:11,385 - mmdet - INFO - Evaluating segm... 2021-05-26 16:52:11,571 - mmdet - INFO - Saving checkpoint at 38 epochs 2021-05-26 16:52:12,258 - mmdet - INFO - Exp name: yolact.py 2021-05-26 16:52:12,258 - mmdet - INFO - Epoch(val) [38][206] bbox_mAP: 0.6170, bbox_mAP_50: 0.8300, bbox_mAP_75: 0.6970, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3380, bbox_mAP_l: 0.6670, bbox_mAP_copypaste: 0.617 0.830 0.697 -1.000 0.338 0.667, segm_mAP: 0.4920, segm_mAP_50: 0.8000, segm_mAP_75: 0.4930, segm_mAP_s: -1.0000, segm_mAP_m: 0.1130, segm_mAP_l: 0.5470, segm_mAP_copypaste: 0.492 0.800 0.493 -1.000 0.113 0.547 2021-05-26 16:55:45,636 - mmdet - INFO - Epoch [39][50/124] lr: 1.000e-04, eta: 1:22:17, time: 4.266, data_time: 3.767, memory: 11791, loss_cls: 0.2241, loss_bbox: 0.1430, loss_segm: 0.0393, loss_mask: 0.8768, loss: 1.2832 2021-05-26 16:59:12,707 - mmdet - INFO - Epoch [39][100/124] lr: 1.000e-04, eta: 1:19:35, time: 4.141, data_time: 3.640, memory: 11791, loss_cls: 0.2141, loss_bbox: 0.1262, loss_segm: 0.0447, loss_mask: 0.8823, loss: 1.2673 2021-05-26 17:01:58,490 - mmdet - INFO - Evaluating bbox... 2021-05-26 17:01:58,648 - mmdet - INFO - Evaluating segm... 2021-05-26 17:01:58,837 - mmdet - INFO - Saving checkpoint at 39 epochs 2021-05-26 17:01:59,497 - mmdet - INFO - Exp name: yolact.py 2021-05-26 17:01:59,498 - mmdet - INFO - Epoch(val) [39][206] bbox_mAP: 0.6070, bbox_mAP_50: 0.8250, bbox_mAP_75: 0.6880, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3370, bbox_mAP_l: 0.6550, bbox_mAP_copypaste: 0.607 0.825 0.688 -1.000 0.337 0.655, segm_mAP: 0.4840, segm_mAP_50: 0.7880, segm_mAP_75: 0.4710, segm_mAP_s: -1.0000, segm_mAP_m: 0.1170, segm_mAP_l: 0.5380, segm_mAP_copypaste: 0.484 0.788 0.471 -1.000 0.117 0.538 2021-05-26 17:05:22,086 - mmdet - INFO - Epoch [40][50/124] lr: 1.000e-04, eta: 1:15:07, time: 4.048, data_time: 3.551, memory: 11791, loss_cls: 0.2478, loss_bbox: 0.1465, loss_segm: 0.0496, loss_mask: 0.8909, loss: 1.3349 2021-05-26 17:09:55,725 - mmdet - INFO - Epoch [40][100/124] lr: 1.000e-04, eta: 1:12:41, time: 5.472, data_time: 4.943, memory: 11791, loss_cls: 0.2722, loss_bbox: 0.1489, loss_segm: 0.0445, loss_mask: 0.8914, loss: 1.3570 2021-05-26 17:13:25,928 - mmdet - INFO - Evaluating bbox... 2021-05-26 17:13:26,094 - mmdet - INFO - Evaluating segm... 2021-05-26 17:13:26,305 - mmdet - INFO - Saving checkpoint at 40 epochs 2021-05-26 17:13:27,049 - mmdet - INFO - Exp name: yolact.py 2021-05-26 17:13:27,049 - mmdet - INFO - Epoch(val) [40][206] bbox_mAP: 0.6230, bbox_mAP_50: 0.8370, bbox_mAP_75: 0.7010, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3250, bbox_mAP_l: 0.6740, bbox_mAP_copypaste: 0.623 0.837 0.701 -1.000 0.325 0.674, segm_mAP: 0.5000, segm_mAP_50: 0.8070, segm_mAP_75: 0.4990, segm_mAP_s: -1.0000, segm_mAP_m: 0.1180, segm_mAP_l: 0.5570, segm_mAP_copypaste: 0.500 0.807 0.499 -1.000 0.118 0.557 2021-05-26 17:17:31,088 - mmdet - INFO - Epoch [41][50/124] lr: 1.000e-04, eta: 1:08:23, time: 4.879, data_time: 4.385, memory: 11791, loss_cls: 0.2391, loss_bbox: 0.1309, loss_segm: 0.0464, loss_mask: 0.9142, loss: 1.3306 2021-05-26 17:21:24,777 - mmdet - INFO - Epoch [41][100/124] lr: 1.000e-04, eta: 1:05:45, time: 4.674, data_time: 4.188, memory: 11791, loss_cls: 0.2741, loss_bbox: 0.1323, loss_segm: 0.0431, loss_mask: 0.8697, loss: 1.3193 2021-05-26 17:24:13,386 - mmdet - INFO - Evaluating bbox... 2021-05-26 17:24:13,511 - mmdet - INFO - Evaluating segm... 2021-05-26 17:24:13,725 - mmdet - INFO - Saving checkpoint at 41 epochs 2021-05-26 17:24:14,441 - mmdet - INFO - Exp name: yolact.py 2021-05-26 17:24:14,441 - mmdet - INFO - Epoch(val) [41][206] bbox_mAP: 0.6250, bbox_mAP_50: 0.8360, bbox_mAP_75: 0.7010, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3330, bbox_mAP_l: 0.6750, bbox_mAP_copypaste: 0.625 0.836 0.701 -1.000 0.333 0.675, segm_mAP: 0.4940, segm_mAP_50: 0.7990, segm_mAP_75: 0.4920, segm_mAP_s: -1.0000, segm_mAP_m: 0.0960, segm_mAP_l: 0.5520, segm_mAP_copypaste: 0.494 0.799 0.492 -1.000 0.096 0.552 2021-05-26 17:27:41,180 - mmdet - INFO - Epoch [42][50/124] lr: 1.000e-04, eta: 1:01:18, time: 4.133, data_time: 3.641, memory: 11791, loss_cls: 0.2780, loss_bbox: 0.1458, loss_segm: 0.0468, loss_mask: 0.8830, loss: 1.3535 2021-05-26 17:31:07,624 - mmdet - INFO - Epoch [42][100/124] lr: 1.000e-04, eta: 0:58:32, time: 4.129, data_time: 3.638, memory: 11791, loss_cls: 0.2062, loss_bbox: 0.1250, loss_segm: 0.0504, loss_mask: 0.8329, loss: 1.2146 2021-05-26 17:34:00,711 - mmdet - INFO - Evaluating bbox... 2021-05-26 17:34:00,870 - mmdet - INFO - Evaluating segm... 2021-05-26 17:34:01,052 - mmdet - INFO - Saving checkpoint at 42 epochs 2021-05-26 17:34:01,806 - mmdet - INFO - Exp name: yolact.py 2021-05-26 17:34:01,807 - mmdet - INFO - Epoch(val) [42][206] bbox_mAP: 0.6180, bbox_mAP_50: 0.8340, bbox_mAP_75: 0.6940, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3270, bbox_mAP_l: 0.6670, bbox_mAP_copypaste: 0.618 0.834 0.694 -1.000 0.327 0.667, segm_mAP: 0.5010, segm_mAP_50: 0.8040, segm_mAP_75: 0.5050, segm_mAP_s: -1.0000, segm_mAP_m: 0.1220, segm_mAP_l: 0.5560, segm_mAP_copypaste: 0.501 0.804 0.505 -1.000 0.122 0.556 2021-05-26 17:37:28,708 - mmdet - INFO - Epoch [43][50/124] lr: 1.000e-05, eta: 0:54:08, time: 4.136, data_time: 3.641, memory: 11791, loss_cls: 0.2415, loss_bbox: 0.1305, loss_segm: 0.0411, loss_mask: 0.8741, loss: 1.2872 2021-05-26 17:41:15,774 - mmdet - INFO - Epoch [43][100/124] lr: 1.000e-05, eta: 0:51:25, time: 4.541, data_time: 4.046, memory: 11791, loss_cls: 0.2519, loss_bbox: 0.1350, loss_segm: 0.0451, loss_mask: 0.9093, loss: 1.3413 2021-05-26 17:43:59,636 - mmdet - INFO - Evaluating bbox... 2021-05-26 17:43:59,791 - mmdet - INFO - Evaluating segm... 2021-05-26 17:44:00,376 - mmdet - INFO - Saving checkpoint at 43 epochs 2021-05-26 17:44:01,072 - mmdet - INFO - Exp name: yolact.py 2021-05-26 17:44:01,072 - mmdet - INFO - Epoch(val) [43][206] bbox_mAP: 0.6220, bbox_mAP_50: 0.8320, bbox_mAP_75: 0.6970, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3350, bbox_mAP_l: 0.6720, bbox_mAP_copypaste: 0.622 0.832 0.697 -1.000 0.335 0.672, segm_mAP: 0.4970, segm_mAP_50: 0.8020, segm_mAP_75: 0.4910, segm_mAP_s: -1.0000, segm_mAP_m: 0.1160, segm_mAP_l: 0.5530, segm_mAP_copypaste: 0.497 0.802 0.491 -1.000 0.116 0.553 2021-05-26 17:47:50,089 - mmdet - INFO - Epoch [44][50/124] lr: 1.000e-05, eta: 0:47:04, time: 4.578, data_time: 4.068, memory: 11791, loss_cls: 0.2437, loss_bbox: 0.1311, loss_segm: 0.0418, loss_mask: 0.9063, loss: 1.3230 2021-05-26 17:51:25,370 - mmdet - INFO - Epoch [44][100/124] lr: 1.000e-05, eta: 0:44:18, time: 4.306, data_time: 3.809, memory: 11791, loss_cls: 0.2076, loss_bbox: 0.1148, loss_segm: 0.0435, loss_mask: 0.8607, loss: 1.2266 2021-05-26 17:54:16,432 - mmdet - INFO - Evaluating bbox... 2021-05-26 17:54:16,585 - mmdet - INFO - Evaluating segm... 2021-05-26 17:54:16,767 - mmdet - INFO - Saving checkpoint at 44 epochs 2021-05-26 17:54:17,495 - mmdet - INFO - Exp name: yolact.py 2021-05-26 17:54:17,495 - mmdet - INFO - Epoch(val) [44][206] bbox_mAP: 0.6270, bbox_mAP_50: 0.8350, bbox_mAP_75: 0.6950, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3230, bbox_mAP_l: 0.6780, bbox_mAP_copypaste: 0.627 0.835 0.695 -1.000 0.323 0.678, segm_mAP: 0.5000, segm_mAP_50: 0.8050, segm_mAP_75: 0.4950, segm_mAP_s: -1.0000, segm_mAP_m: 0.1130, segm_mAP_l: 0.5560, segm_mAP_copypaste: 0.500 0.805 0.495 -1.000 0.113 0.556 2021-05-26 17:57:54,614 - mmdet - INFO - Epoch [45][50/124] lr: 1.000e-05, eta: 0:39:57, time: 4.340, data_time: 3.841, memory: 11791, loss_cls: 0.2337, loss_bbox: 0.1219, loss_segm: 0.0403, loss_mask: 0.8456, loss: 1.2415 2021-05-26 18:01:43,782 - mmdet - INFO - Epoch [45][100/124] lr: 1.000e-05, eta: 0:37:11, time: 4.583, data_time: 4.089, memory: 11791, loss_cls: 0.2413, loss_bbox: 0.1338, loss_segm: 0.0430, loss_mask: 0.8843, loss: 1.3025 2021-05-26 18:04:39,754 - mmdet - INFO - Evaluating bbox... 2021-05-26 18:04:39,878 - mmdet - INFO - Evaluating segm... 2021-05-26 18:04:40,082 - mmdet - INFO - Saving checkpoint at 45 epochs 2021-05-26 18:04:40,760 - mmdet - INFO - Exp name: yolact.py 2021-05-26 18:04:40,761 - mmdet - INFO - Epoch(val) [45][206] bbox_mAP: 0.6200, bbox_mAP_50: 0.8280, bbox_mAP_75: 0.6880, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3260, bbox_mAP_l: 0.6710, bbox_mAP_copypaste: 0.620 0.828 0.688 -1.000 0.326 0.671, segm_mAP: 0.4940, segm_mAP_50: 0.7990, segm_mAP_75: 0.4920, segm_mAP_s: -1.0000, segm_mAP_m: 0.1190, segm_mAP_l: 0.5500, segm_mAP_copypaste: 0.494 0.799 0.492 -1.000 0.119 0.550 2021-05-26 18:08:20,722 - mmdet - INFO - Epoch [46][50/124] lr: 1.000e-05, eta: 0:32:51, time: 4.397, data_time: 3.896, memory: 11791, loss_cls: 0.2535, loss_bbox: 0.1415, loss_segm: 0.0436, loss_mask: 0.9078, loss: 1.3465 2021-05-26 18:11:52,510 - mmdet - INFO - Epoch [46][100/124] lr: 1.000e-05, eta: 0:30:01, time: 4.236, data_time: 3.735, memory: 11791, loss_cls: 0.2262, loss_bbox: 0.1159, loss_segm: 0.0370, loss_mask: 0.8533, loss: 1.2323 2021-05-26 18:14:47,575 - mmdet - INFO - Evaluating bbox... 2021-05-26 18:14:47,700 - mmdet - INFO - Evaluating segm... 2021-05-26 18:14:47,888 - mmdet - INFO - Saving checkpoint at 46 epochs 2021-05-26 18:14:48,578 - mmdet - INFO - Exp name: yolact.py 2021-05-26 18:14:48,578 - mmdet - INFO - Epoch(val) [46][206] bbox_mAP: 0.6180, bbox_mAP_50: 0.8290, bbox_mAP_75: 0.6900, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3230, bbox_mAP_l: 0.6680, bbox_mAP_copypaste: 0.618 0.829 0.690 -1.000 0.323 0.668, segm_mAP: 0.4950, segm_mAP_50: 0.7990, segm_mAP_75: 0.4910, segm_mAP_s: -1.0000, segm_mAP_m: 0.1150, segm_mAP_l: 0.5500, segm_mAP_copypaste: 0.495 0.799 0.491 -1.000 0.115 0.550 2021-05-26 18:18:13,778 - mmdet - INFO - Epoch [47][50/124] lr: 1.000e-05, eta: 0:25:41, time: 4.102, data_time: 3.612, memory: 11791, loss_cls: 0.2589, loss_bbox: 0.1373, loss_segm: 0.0376, loss_mask: 0.8904, loss: 1.3242 2021-05-26 18:21:45,891 - mmdet - INFO - Epoch [47][100/124] lr: 1.000e-05, eta: 0:22:51, time: 4.242, data_time: 3.744, memory: 11791, loss_cls: 0.2568, loss_bbox: 0.1505, loss_segm: 0.0416, loss_mask: 0.8897, loss: 1.3386 2021-05-26 18:24:13,637 - mmdet - INFO - Evaluating bbox... 2021-05-26 18:24:13,764 - mmdet - INFO - Evaluating segm... 2021-05-26 18:24:13,979 - mmdet - INFO - Saving checkpoint at 47 epochs 2021-05-26 18:24:14,686 - mmdet - INFO - Exp name: yolact.py 2021-05-26 18:24:14,686 - mmdet - INFO - Epoch(val) [47][206] bbox_mAP: 0.6190, bbox_mAP_50: 0.8310, bbox_mAP_75: 0.6910, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3080, bbox_mAP_l: 0.6710, bbox_mAP_copypaste: 0.619 0.831 0.691 -1.000 0.308 0.671, segm_mAP: 0.4970, segm_mAP_50: 0.7950, segm_mAP_75: 0.4950, segm_mAP_s: -1.0000, segm_mAP_m: 0.1120, segm_mAP_l: 0.5520, segm_mAP_copypaste: 0.497 0.795 0.495 -1.000 0.112 0.552 2021-05-26 18:27:40,684 - mmdet - INFO - Epoch [48][50/124] lr: 1.000e-05, eta: 0:18:32, time: 4.118, data_time: 3.620, memory: 11791, loss_cls: 0.2308, loss_bbox: 0.1323, loss_segm: 0.0458, loss_mask: 0.8830, loss: 1.2920 2021-05-26 18:31:21,465 - mmdet - INFO - Epoch [48][100/124] lr: 1.000e-05, eta: 0:15:41, time: 4.415, data_time: 3.913, memory: 11791, loss_cls: 0.2365, loss_bbox: 0.1351, loss_segm: 0.0401, loss_mask: 0.8755, loss: 1.2872 2021-05-26 18:34:08,613 - mmdet - INFO - Evaluating bbox... 2021-05-26 18:34:08,740 - mmdet - INFO - Evaluating segm... 2021-05-26 18:34:08,963 - mmdet - INFO - Saving checkpoint at 48 epochs 2021-05-26 18:34:09,686 - mmdet - INFO - Exp name: yolact.py 2021-05-26 18:34:09,687 - mmdet - INFO - Epoch(val) [48][206] bbox_mAP: 0.6200, bbox_mAP_50: 0.8300, bbox_mAP_75: 0.6890, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3210, bbox_mAP_l: 0.6710, bbox_mAP_copypaste: 0.620 0.830 0.689 -1.000 0.321 0.671, segm_mAP: 0.4940, segm_mAP_50: 0.8000, segm_mAP_75: 0.4890, segm_mAP_s: -1.0000, segm_mAP_m: 0.1120, segm_mAP_l: 0.5500, segm_mAP_copypaste: 0.494 0.800 0.489 -1.000 0.112 0.550 2021-05-26 18:37:42,455 - mmdet - INFO - Epoch [49][50/124] lr: 1.000e-05, eta: 0:11:24, time: 4.253, data_time: 3.763, memory: 11791, loss_cls: 0.2156, loss_bbox: 0.1252, loss_segm: 0.0425, loss_mask: 0.8556, loss: 1.2389 2021-05-26 18:41:29,050 - mmdet - INFO - Epoch [49][100/124] lr: 1.000e-05, eta: 0:08:32, time: 4.532, data_time: 4.040, memory: 11791, loss_cls: 0.1995, loss_bbox: 0.1156, loss_segm: 0.0387, loss_mask: 0.8337, loss: 1.1875 2021-05-26 18:44:09,747 - mmdet - INFO - Evaluating bbox... 2021-05-26 18:44:09,911 - mmdet - INFO - Evaluating segm... 2021-05-26 18:44:10,104 - mmdet - INFO - Saving checkpoint at 49 epochs 2021-05-26 18:44:10,819 - mmdet - INFO - Exp name: yolact.py 2021-05-26 18:44:10,819 - mmdet - INFO - Epoch(val) [49][206] bbox_mAP: 0.6220, bbox_mAP_50: 0.8300, bbox_mAP_75: 0.6900, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3220, bbox_mAP_l: 0.6730, bbox_mAP_copypaste: 0.622 0.830 0.690 -1.000 0.322 0.673, segm_mAP: 0.4970, segm_mAP_50: 0.8000, segm_mAP_75: 0.4930, segm_mAP_s: -1.0000, segm_mAP_m: 0.1140, segm_mAP_l: 0.5530, segm_mAP_copypaste: 0.497 0.800 0.493 -1.000 0.114 0.553 2021-05-26 18:47:55,546 - mmdet - INFO - Epoch [50][50/124] lr: 1.000e-06, eta: 0:04:15, time: 4.492, data_time: 4.001, memory: 11791, loss_cls: 0.2803, loss_bbox: 0.1453, loss_segm: 0.0402, loss_mask: 0.9003, loss: 1.3661 2021-05-26 18:51:14,656 - mmdet - INFO - Epoch [50][100/124] lr: 1.000e-06, eta: 0:01:23, time: 3.982, data_time: 3.487, memory: 11791, loss_cls: 0.2010, loss_bbox: 0.1065, loss_segm: 0.0475, loss_mask: 0.8397, loss: 1.1948 2021-05-26 18:54:09,329 - mmdet - INFO - Evaluating bbox... 2021-05-26 18:54:09,477 - mmdet - INFO - Evaluating segm... 2021-05-26 18:54:10,125 - mmdet - INFO - Saving checkpoint at 50 epochs 2021-05-26 18:54:10,906 - mmdet - INFO - Exp name: yolact.py 2021-05-26 18:54:10,906 - mmdet - INFO - Epoch(val) [50][206] bbox_mAP: 0.6210, bbox_mAP_50: 0.8310, bbox_mAP_75: 0.6910, bbox_mAP_s: -1.0000, bbox_mAP_m: 0.3230, bbox_mAP_l: 0.6720, bbox_mAP_copypaste: 0.621 0.831 0.691 -1.000 0.323 0.672, segm_mAP: 0.4970, segm_mAP_50: 0.8010, segm_mAP_75: 0.4940, segm_mAP_s: -1.0000, segm_mAP_m: 0.1140, segm_mAP_l: 0.5530, segm_mAP_copypaste: 0.497 0.801 0.494 -1.000 0.114 0.553

olfa-koubaa commented 3 years ago

Are you sure you are loading a checkpoint to speed up training? How many images are you using when training?

yes I am I'm using 987 images for training which is not a lot so it shouldn't slow the training this much

jshilong commented 3 years ago

Different from Detectron2, In our implementation, dataloader would relaunch after every epoch, It would cost much time when your train a small dataset, We recommend you use RepeatDataset to build a larger dataset instead of setting a larger epoch number.

jshilong commented 3 years ago

We should add this feature to a future version. @RangiLyu @ZwwWayne @hhaAndroid

olfa-koubaa commented 3 years ago

Different from Detectron2, In our implementation, dataloader would relaunch after every epoch, It would cost much time when your train a small dataset, We recommend you use RepeatDataset to build a larger dataset instead of setting a larger epoch number.

Thank you for your response

I'm using repeat dataset now in the traininig,validation and test datasets but I'm getting this error:

'RepeatDataset' object has no attribute 'evaluate'

should I use repeat dataset only in the training dataset ?

RangiLyu commented 3 years ago

Yes, you only need to use RepeatDataset in your training dataset.