search

hasanirtiza / Pedestron

[Pedestron] Generalizable Pedestrian Detection: The Elephant In The Room. @ CVPR2021
https://openaccess.thecvf.com/content/CVPR2021/papers/Hasan_Generalizable_Pedestrian_Detection_The_Elephant_in_the_Room_CVPR_2021_paper.pdf
Apache License 2.0
687 stars 159 forks source link

Testing problem #66

Closed msha096 closed 4 years ago

msha096 commented 4 years ago

I am facing the following error when I eval the pre-trained model on CityPersons dataset.

  File "./tools/test_city_person.py", line 230, in <module>
    main()
  File "./tools/test_city_person.py", line 222, in main
    MRs = validate('datasets/CityPersons/val_gt.json', args.out)
  File "/home/mingzhi/Downloads/Pedestron/tools/cityPerson/eval_demo.py", line 13, in validate
    cocoEval = COCOeval(cocoGt, cocoDt, 'bbox')
  File "/home/mingzhi/Downloads/Pedestron/tools/cityPerson/eval_MR_multisetup.py", line 76, in __init__
    self.params = Params(iouType=iouType) # parameters
  File "/home/mingzhi/Downloads/Pedestron/tools/cityPerson/eval_MR_multisetup.py", line 525, in __init__
    self.setDetParams()
  File "/home/mingzhi/Downloads/Pedestron/tools/cityPerson/eval_MR_multisetup.py", line 501, in setDetParams
    self.recThrs = np.linspace(.0, 1.00, np.round((1.00 - .0) / .01) + 1, endpoint=True)
  File "<__array_function__ internals>", line 6, in linspace
  File "/home/mingzhi/anaconda3/envs/pedestron/lib/python3.7/site-packages/numpy/core/function_base.py", line 113, in linspace
    num = operator.index(num)
TypeError: 'numpy.float64' object cannot be interpreted as an integer

When I train the model and eval my own trained models, I am having this problem: File "./tools/test_city_person.py", line 230, in <module> main() File "./tools/test_city_person.py", line 222, in main MRs = validate('datasets/CityPersons/val_gt.json', args.out) File "/home/mingzhi/Downloads/Pedestron/tools/cityPerson/eval_demo.py", line 11, in validate cocoDt = cocoGt.loadRes(dt_path) File "/home/mingzhi/Downloads/Pedestron/tools/cityPerson/coco.py", line 313, in loadRes if 'caption' in anns[0]: IndexError: list index out of range

How can I solve the above two problems?? I know for the second one, it seems the model does not detect, but why?

hasanirtiza commented 4 years ago

Regarding the first problem, it is related to your numpy version. Try to test it with numpy == 1.14.3 or below. Secondly, test you pre-trained model on demo images, see if it works. Yes, the problem is saying that your model is not able to detect anything so probably something went wrong during training. Hard to comment what went wrong but something did.

msha096 commented 4 years ago

test you pre-trained model on demo images, see if it works. Yes, the problem is saying that your model is not able to detect anything so probably something went wrong during training. Hard to comment

NumPy 1.14.3 has conflict with pytorch 1.2, I tried NumPy 1.11.3 by conda install -c conda-forge numpy==1.11.3 and conda install numpy==1.11.3. I tried on the current conda environment and I even create a new conda environment and re-install everything, but it causes a new problem:

RuntimeError: module compiled against API version 0xc but this version of numpy is 0xa
Traceback (most recent call last):
  File "./tools/test_city_person.py", line 9, in <module>
    import mmcv
  File "/home/mingzhi/anaconda3/envs/pedest/lib/python3.7/site-packages/mmcv/__init__.py", line 5, in <module>
    from .opencv_info import *
  File "/home/mingzhi/anaconda3/envs/pedest/lib/python3.7/site-packages/mmcv/opencv_info.py", line 1, in <module>
    import cv2
  File "/home/mingzhi/anaconda3/envs/pedest/lib/python3.7/site-packages/cv2/__init__.py", line 5, in <module>
    from .cv2 import *
ImportError: numpy.core.multiarray failed to import
msha096 commented 4 years ago

Numpy problem solution update: Just in case someone has the same problem with NumPy version. You could either downgrade NumPy to lower version (lower than 1.18) (I failed with this solution) To use with NumPy >=1.18, you could try pip install -U 'git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI'(this solution does not work for me but it works for someone using MMDetection)

The solution works for me is: LOC 501 of the file /Pedestron/tools/cityPerson/eval_MR_multisetup.py, change self.recThrs = np.linspace(.0, 1.00, np.round((1.00 - .0) / .01) + 1, endpoint=True) into self.recThrs = np.linspace(.0, 1.00, 101, endpoint=True)

Cheers

msha096 commented 4 years ago

Regarding the first problem, it is related to your numpy version. Try to test it with numpy == 1.14.3 or below. Secondly, test you pre-trained model on demo images, see if it works. Yes, the problem is saying that your model is not able to detect anything so probably something went wrong during training. Hard to comment what went wrong but something did.

I eval my mode and it does not predict anything. Here are the config file and training log. It seems it is converging but maybe I need to train more epochs? Can I know how many epochs did you train to get around 14MR on the CityPersons dataset with ResNet backbone.

model = dict(
    type='RetinaNet',
    pretrained='modelzoo://resnet50', 
    backbone=dict(
        type='ResNet',
        depth=50,
        #groups=64,
        #base_width=4,
        num_stages=4,
        out_indices=(0, 1, 2, 3),
        frozen_stages=1,
        style='pytorch'),
    neck=dict(
        type='FPN',
        in_channels=[256, 512, 1024, 2048],
        out_channels=256,
        start_level=1,
        add_extra_convs=True,
        num_outs=5),
    bbox_head=dict(
        type='RetinaHead',
        num_classes=81, # change to 2??
        in_channels=256,
        stacked_convs=4,
        feat_channels=256,
        octave_base_scale=4,
        scales_per_octave=3,
        anchor_ratios=[0.5, 1.0, 2.0],
        anchor_strides=[8, 16, 32, 64, 128],
        target_means=[.0, .0, .0, .0],
        target_stds=[1.0, 1.0, 1.0, 1.0],
        loss_cls=dict(
            type='FocalLoss',
            use_sigmoid=True,
            gamma=2.0,
            alpha=0.25,
            loss_weight=1.0),
        loss_bbox=dict(type='SmoothL1Loss', beta=0.11, loss_weight=1.0)))
# training and testing settings
train_cfg = dict(
    assigner=dict(
        type='MaxIoUAssigner',
        pos_iou_thr=0.5,
        neg_iou_thr=0.4,
        min_pos_iou=0,
        ignore_iof_thr=-1),
    allowed_border=-1,
    pos_weight=-1,
    debug=False)
test_cfg = dict(
    nms_pre=1000,
    min_bbox_size=0,
    score_thr=0.05,
    nms=dict(type='nms', iou_thr=0.5),
    max_per_img=100)
# dataset settings
dataset_type = 'CocoDataset'
data_root = 'datasets/CityPersons/'
img_norm_cfg = dict(
    mean=[123.675, 116.28, 103.53],
    std=[58.395, 57.12, 57.375],
    to_rgb=True)
data = dict(
    imgs_per_gpu=2, # 2020-09-05
    workers_per_gpu=5,
    train=dict(
        type=dataset_type,
    ann_file=data_root + 'annotations/train.json',
    img_prefix=data_root,
    #img_scale=[(1216, 608),(2048, 1024)],
    img_scale=[(960, 480),(2048, 1024)],
        multiscale_mode='range',
        img_norm_cfg=img_norm_cfg,
        size_divisor=32,
        flip_ratio=0.5,
        with_mask=False,
        with_crowd=True,
        with_label=True,
        extra_aug=dict(
            photo_metric_distortion=dict(brightness_delta=180, contrast_range=(0.5, 1.5),
                 saturation_range=(0.5, 1.5), hue_delta=18),
             random_crop=dict(min_ious=(0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9), min_crop_size=0.1),
         ),
    ),
    test=dict(
        type=dataset_type,
    ann_file=data_root + 'annotations/val_gt_for_mmdetction.json',
        #img_prefix=data_root,
        img_prefix=data_root + '/leftImg8bit_trainvaltest/leftImg8bit/val_all_in_folder/',
        img_scale=(2048, 1024),
        img_norm_cfg=img_norm_cfg,
        size_divisor=32,
        flip_ratio=0,
        with_mask=False,
        with_label=False,
        test_mode=True))
# optimizer
mean_teacher=True
optimizer = dict(type='SGD', lr=0.01, momentum=0.9, weight_decay=0.0001)
optimizer_config = dict(grad_clip=dict(max_norm=35, norm_type=2), mean_teacher = dict(alpha=0.999))
# learning policy
lr_config = dict(
    policy='step',
    warmup='linear',
    warmup_iters=500,
    warmup_ratio=1.0 / 3,
    step=[8, 11])
checkpoint_config = dict(interval=1)
# yapf:disable
log_config = dict(
    interval=50,
    hooks=[
        dict(type='TextLoggerHook'),
        # dict(type='TensorboardLoggerHook')
    ])
# yapf:enable
# runtime settings
total_epochs = 12
device_ids = range(1) # change from 8 to 1, 2020-09-05
dist_params = dict(backend='nccl')
log_level = 'INFO'
work_dir = './work_dirs/retinanet_res50'
load_from = None
resume_from = None
workflow = [('train', 1)]

2020-09-06 14:23:43,787 - INFO - workflow: [('train', 1)], max: 12 epochs 2020-09-06 14:24:05,140 - INFO - Epoch [1][50/1389] lr: 0.00399, eta: 1:58:16, time: 0.427, data_time: 0.015, memory: 6089, loss_cls: 1.4356, loss_bbox: 0.3681, loss: 1.8037 2020-09-06 14:24:25,832 - INFO - Epoch [1][100/1389] lr: 0.00465, eta: 1:56:05, time: 0.414, data_time: 0.004, memory: 6470, loss_cls: 1.3702, loss_bbox: 0.3726, loss: 1.7427 2020-09-06 14:24:46,772 - INFO - Epoch [1][150/1389] lr: 0.00532, eta: 1:55:35, time: 0.419, data_time: 0.004, memory: 6910, loss_cls: 1.2591, loss_bbox: 0.3587, loss: 1.6178 2020-09-06 14:25:07,427 - INFO - Epoch [1][200/1389] lr: 0.00599, eta: 1:54:46, time: 0.413, data_time: 0.004, memory: 8041, loss_cls: 1.1597, loss_bbox: 0.3679, loss: 1.5277 2020-09-06 14:25:27,596 - INFO - Epoch [1][250/1389] lr: 0.00665, eta: 1:53:37, time: 0.403, data_time: 0.004, memory: 8041, loss_cls: 1.1086, loss_bbox: 0.3761, loss: 1.4846 2020-09-06 14:25:45,663 - INFO - Epoch [1][300/1389] lr: 0.00732, eta: 1:50:49, time: 0.361, data_time: 0.003, memory: 8041, loss_cls: 0.9605, loss_bbox: 0.3561, loss: 1.3166 2020-09-06 14:26:04,877 - INFO - Epoch [1][350/1389] lr: 0.00799, eta: 1:49:37, time: 0.384, data_time: 0.003, memory: 8041, loss_cls: 41.0122, loss_bbox: 0.3567, loss: 41.3689 2020-09-06 14:26:24,233 - INFO - Epoch [1][400/1389] lr: 0.00865, eta: 1:48:45, time: 0.387, data_time: 0.003, memory: 8041, loss_cls: 1.0285, loss_bbox: 0.3607, loss: 1.3892 2020-09-06 14:26:43,899 - INFO - Epoch [1][450/1389] lr: 0.00932, eta: 1:48:11, time: 0.393, data_time: 0.004, memory: 8041, loss_cls: 1.1816, loss_bbox: 0.3805, loss: 1.5621 2020-09-06 14:27:02,592 - INFO - Epoch [1][500/1389] lr: 0.00999, eta: 1:47:08, time: 0.374, data_time: 0.003, memory: 8041, loss_cls: 1.0417, loss_bbox: 0.3691, loss: 1.4108 2020-09-06 14:27:21,898 - INFO - Epoch [1][550/1389] lr: 0.01000, eta: 1:46:31, time: 0.386, data_time: 0.003, memory: 8041, loss_cls: 1.0428, loss_bbox: 0.3709, loss: 1.4137 2020-09-06 14:27:41,885 - INFO - Epoch [1][600/1389] lr: 0.01000, eta: 1:46:16, time: 0.400, data_time: 0.004, memory: 8041, loss_cls: 100.6854, loss_bbox: 0.3730, loss: 101.0584 2020-09-06 14:28:02,809 - INFO - Epoch [1][650/1389] lr: 0.01000, eta: 1:46:23, time: 0.418, data_time: 0.004, memory: 8041, loss_cls: 1.0212, loss_bbox: 0.3750, loss: 1.3962 2020-09-06 14:28:23,175 - INFO - Epoch [1][700/1389] lr: 0.01000, eta: 1:46:13, time: 0.407, data_time: 0.004, memory: 8041, loss_cls: 1.0790, loss_bbox: 0.3805, loss: 1.4596 2020-09-06 14:28:43,148 - INFO - Epoch [1][750/1389] lr: 0.01000, eta: 1:45:53, time: 0.399, data_time: 0.004, memory: 8041, loss_cls: 0.9720, loss_bbox: 0.3679, loss: 1.3399 2020-09-06 14:29:03,753 - INFO - Epoch [1][800/1389] lr: 0.01000, eta: 1:45:46, time: 0.412, data_time: 0.004, memory: 8041, loss_cls: 1.0244, loss_bbox: 0.3684, loss: 1.3928 2020-09-06 14:29:22,866 - INFO - Epoch [1][850/1389] lr: 0.01000, eta: 1:45:09, time: 0.382, data_time: 0.004, memory: 8041, loss_cls: 0.8932, loss_bbox: 0.3576, loss: 1.2507 2020-09-06 14:29:41,915 - INFO - Epoch [1][900/1389] lr: 0.01000, eta: 1:44:34, time: 0.381, data_time: 0.004, memory: 8041, loss_cls: 0.8793, loss_bbox: 0.3683, loss: 1.2477 2020-09-06 14:30:00,845 - INFO - Epoch [1][950/1389] lr: 0.01000, eta: 1:43:58, time: 0.379, data_time: 0.003, memory: 8041, loss_cls: 0.9031, loss_bbox: 0.3655, loss: 1.2686 2020-09-06 14:30:20,748 - INFO - Epoch [1][1000/1389] lr: 0.01000, eta: 1:43:39, time: 0.398, data_time: 0.004, memory: 8041, loss_cls: 1.0284, loss_bbox: 0.3706, loss: 1.3990 2020-09-06 14:30:40,299 - INFO - Epoch [1][1050/1389] lr: 0.01000, eta: 1:43:15, time: 0.391, data_time: 0.004, memory: 8041, loss_cls: 0.9209, loss_bbox: 0.3620, loss: 1.2829 2020-09-06 14:31:01,716 - INFO - Epoch [1][1100/1389] lr: 0.01000, eta: 1:43:17, time: 0.428, data_time: 0.004, memory: 8041, loss_cls: 1.0773, loss_bbox: 0.3650, loss: 1.4423 2020-09-06 14:31:20,524 - INFO - Epoch [1][1150/1389] lr: 0.01000, eta: 1:42:43, time: 0.376, data_time: 0.003, memory: 8041, loss_cls: 0.8656, loss_bbox: 0.3599, loss: 1.2255 2020-09-06 14:31:40,491 - INFO - Epoch [1][1200/1389] lr: 0.01000, eta: 1:42:24, time: 0.399, data_time: 0.004, memory: 8041, loss_cls: 0.9228, loss_bbox: 0.3636, loss: 1.2864 2020-09-06 14:32:00,378 - INFO - Epoch [1][1250/1389] lr: 0.01000, eta: 1:42:05, time: 0.398, data_time: 0.004, memory: 8041, loss_cls: 0.9609, loss_bbox: 0.3773, loss: 1.3382 2020-09-06 14:32:20,376 - INFO - Epoch [1][1300/1389] lr: 0.01000, eta: 1:41:46, time: 0.400, data_time: 0.004, memory: 8041, loss_cls: 0.9193, loss_bbox: 0.3747, loss: 1.2940 2020-09-06 14:32:39,400 - INFO - Epoch [1][1350/1389] lr: 0.01000, eta: 1:41:17, time: 0.380, data_time: 0.004, memory: 8041, loss_cls: 0.8903, loss_bbox: 0.3619, loss: 1.2521 2020-09-06 14:33:14,448 - INFO - Epoch [2][50/1389] lr: 0.01000, eta: 1:38:03, time: 0.407, data_time: 0.014, memory: 8041, loss_cls: 0.9373, loss_bbox: 0.3719, loss: 1.3092 2020-09-06 14:33:34,591 - INFO - Epoch [2][100/1389] lr: 0.01000, eta: 1:37:52, time: 0.403, data_time: 0.003, memory: 8041, loss_cls: 0.9136, loss_bbox: 0.3757, loss: 1.2893 2020-09-06 14:33:54,090 - INFO - Epoch [2][150/1389] lr: 0.01000, eta: 1:37:35, time: 0.390, data_time: 0.003, memory: 8041, loss_cls: 0.8521, loss_bbox: 0.3580, loss: 1.2101 2020-09-06 14:34:13,655 - INFO - Epoch [2][200/1389] lr: 0.01000, eta: 1:37:17, time: 0.391, data_time: 0.003, memory: 8041, loss_cls: 0.8546, loss_bbox: 0.3725, loss: 1.2271 2020-09-06 14:34:34,086 - INFO - Epoch [2][250/1389] lr: 0.01000, eta: 1:37:08, time: 0.409, data_time: 0.004, memory: 8041, loss_cls: 0.9733, loss_bbox: 0.3559, loss: 1.3292 2020-09-06 14:34:53,485 - INFO - Epoch [2][300/1389] lr: 0.01000, eta: 1:36:48, time: 0.388, data_time: 0.003, memory: 8041, loss_cls: 0.8379, loss_bbox: 0.3627, loss: 1.2007 2020-09-06 14:35:13,555 - INFO - Epoch [2][350/1389] lr: 0.01000, eta: 1:36:35, time: 0.401, data_time: 0.004, memory: 8041, loss_cls: 0.8373, loss_bbox: 0.3629, loss: 1.2002 2020-09-06 14:35:33,659 - INFO - Epoch [2][400/1389] lr: 0.01000, eta: 1:36:21, time: 0.402, data_time: 0.004, memory: 8041, loss_cls: 0.9187, loss_bbox: 0.3685, loss: 1.2872 2020-09-06 14:35:53,822 - INFO - Epoch [2][450/1389] lr: 0.01000, eta: 1:36:08, time: 0.403, data_time: 0.004, memory: 8041, loss_cls: 0.8655, loss_bbox: 0.3563, loss: 1.2218 2020-09-06 14:36:13,099 - INFO - Epoch [2][500/1389] lr: 0.01000, eta: 1:35:47, time: 0.386, data_time: 0.003, memory: 9389, loss_cls: 0.8387, loss_bbox: 0.3725, loss: 1.2113 2020-09-06 14:36:33,400 - INFO - Epoch [2][550/1389] lr: 0.01000, eta: 1:35:34, time: 0.406, data_time: 0.004, memory: 9389, loss_cls: 0.8529, loss_bbox: 0.3602, loss: 1.2131 2020-09-06 14:36:53,352 - INFO - Epoch [2][600/1389] lr: 0.01000, eta: 1:35:18, time: 0.399, data_time: 0.004, memory: 9389, loss_cls: 0.8863, loss_bbox: 0.3678, loss: 1.2542 2020-09-06 14:37:13,282 - INFO - Epoch [2][650/1389] lr: 0.01000, eta: 1:35:02, time: 0.399, data_time: 0.004, memory: 9389, loss_cls: 0.8427, loss_bbox: 0.3588, loss: 1.2016 2020-09-06 14:37:32,928 - INFO - Epoch [2][700/1389] lr: 0.01000, eta: 1:34:43, time: 0.393, data_time: 0.004, memory: 9389, loss_cls: 0.9587, loss_bbox: 0.3620, loss: 1.3207 2020-09-06 14:37:53,842 - INFO - Epoch [2][750/1389] lr: 0.01000, eta: 1:34:34, time: 0.418, data_time: 0.004, memory: 9389, loss_cls: 0.8670, loss_bbox: 0.3650, loss: 1.2320 2020-09-06 14:38:12,742 - INFO - Epoch [2][800/1389] lr: 0.01000, eta: 1:34:10, time: 0.378, data_time: 0.004, memory: 9389, loss_cls: 0.8603, loss_bbox: 0.3628, loss: 1.2231 2020-09-06 14:38:34,615 - INFO - Epoch [2][850/1389] lr: 0.01000, eta: 1:34:06, time: 0.437, data_time: 0.004, memory: 9389, loss_cls: 0.8937, loss_bbox: 0.3712, loss: 1.2649 2020-09-06 14:38:54,248 - INFO - Epoch [2][900/1389] lr: 0.01000, eta: 1:33:46, time: 0.393, data_time: 0.004, memory: 9389, loss_cls: 0.8864, loss_bbox: 0.3647, loss: 1.2511 2020-09-06 14:39:13,919 - INFO - Epoch [2][950/1389] lr: 0.01000, eta: 1:33:28, time: 0.393, data_time: 0.004, memory: 9389, loss_cls: 0.8422, loss_bbox: 0.3586, loss: 1.2007 2020-09-06 14:39:32,939 - INFO - Epoch [2][1000/1389] lr: 0.01000, eta: 1:33:05, time: 0.380, data_time: 0.004, memory: 9389, loss_cls: 0.8521, loss_bbox: 0.3668, loss: 1.2189 2020-09-06 14:39:52,659 - INFO - Epoch [2][1050/1389] lr: 0.01000, eta: 1:32:46, time: 0.394, data_time: 0.004, memory: 9389, loss_cls: 0.8681, loss_bbox: 0.3580, loss: 1.2261 2020-09-06 14:40:13,271 - INFO - Epoch [2][1100/1389] lr: 0.01000, eta: 1:32:33, time: 0.412, data_time: 0.004, memory: 9389, loss_cls: 0.8721, loss_bbox: 0.3561, loss: 1.2282 2020-09-06 14:40:33,573 - INFO - Epoch [2][1150/1389] lr: 0.01000, eta: 1:32:17, time: 0.406, data_time: 0.004, memory: 9389, loss_cls: 0.8557, loss_bbox: 0.3621, loss: 1.2178 2020-09-06 14:40:53,785 - INFO - Epoch [2][1200/1389] lr: 0.01000, eta: 1:32:01, time: 0.404, data_time: 0.004, memory: 9389, loss_cls: 0.8347, loss_bbox: 0.3669, loss: 1.2016 2020-09-06 14:41:15,440 - INFO - Epoch [2][1250/1389] lr: 0.01000, eta: 1:31:52, time: 0.433, data_time: 0.004, memory: 9389, loss_cls: 0.8656, loss_bbox: 0.3605, loss: 1.2261 2020-09-06 14:41:34,009 - INFO - Epoch [2][1300/1389] lr: 0.01000, eta: 1:31:27, time: 0.371, data_time: 0.004, memory: 9389, loss_cls: 0.8651, loss_bbox: 0.3696, loss: 1.2347 2020-09-06 14:41:54,067 - INFO - Epoch [2][1350/1389] lr: 0.01000, eta: 1:31:09, time: 0.401, data_time: 0.004, memory: 9389, loss_cls: 1.2164, loss_bbox: 0.3821, loss: 1.5985 2020-09-06 14:42:29,013 - INFO - Epoch [3][50/1389] lr: 0.01000, eta: 1:29:15, time: 0.375, data_time: 0.013, memory: 9389, loss_cls: 0.8668, loss_bbox: 0.3535, loss: 1.2204 2020-09-06 14:42:49,051 - INFO - Epoch [3][100/1389] lr: 0.01000, eta: 1:28:59, time: 0.401, data_time: 0.004, memory: 9389, loss_cls: 1.0918, loss_bbox: 0.3594, loss: 1.4512 2020-09-06 14:43:09,155 - INFO - Epoch [3][150/1389] lr: 0.01000, eta: 1:28:43, time: 0.402, data_time: 0.003, memory: 9389, loss_cls: 1.0098, loss_bbox: 0.3739, loss: 1.3837 2020-09-06 14:43:29,364 - INFO - Epoch [3][200/1389] lr: 0.01000, eta: 1:28:28, time: 0.404, data_time: 0.004, memory: 9389, loss_cls: 0.8870, loss_bbox: 0.3589, loss: 1.2459 2020-09-06 14:43:50,357 - INFO - Epoch [3][250/1389] lr: 0.01000, eta: 1:28:15, time: 0.420, data_time: 0.004, memory: 9389, loss_cls: 0.8586, loss_bbox: 0.3573, loss: 1.2158 2020-09-06 14:44:10,192 - INFO - Epoch [3][300/1389] lr: 0.01000, eta: 1:27:58, time: 0.397, data_time: 0.004, memory: 9389, loss_cls: 0.8804, loss_bbox: 0.3735, loss: 1.2539 2020-09-06 14:44:29,232 - INFO - Epoch [3][350/1389] lr: 0.01000, eta: 1:27:37, time: 0.381, data_time: 0.003, memory: 9389, loss_cls: 0.8702, loss_bbox: 0.3631, loss: 1.2333 2020-09-06 14:44:49,452 - INFO - Epoch [3][400/1389] lr: 0.01000, eta: 1:27:21, time: 0.404, data_time: 0.004, memory: 9389, loss_cls: 0.8536, loss_bbox: 0.3664, loss: 1.2200 2020-09-06 14:45:09,601 - INFO - Epoch [3][450/1389] lr: 0.01000, eta: 1:27:04, time: 0.403, data_time: 0.003, memory: 9389, loss_cls: 0.8364, loss_bbox: 0.3545, loss: 1.1909 2020-09-06 14:45:29,020 - INFO - Epoch [3][500/1389] lr: 0.01000, eta: 1:26:45, time: 0.388, data_time: 0.004, memory: 9389, loss_cls: 0.8462, loss_bbox: 0.3614, loss: 1.2076 2020-09-06 14:45:48,315 - INFO - Epoch [3][550/1389] lr: 0.01000, eta: 1:26:25, time: 0.386, data_time: 0.004, memory: 9389, loss_cls: 0.8751, loss_bbox: 0.3633, loss: 1.2385 2020-09-06 14:46:08,069 - INFO - Epoch [3][600/1389] lr: 0.01000, eta: 1:26:07, time: 0.395, data_time: 0.004, memory: 9389, loss_cls: 0.8474, loss_bbox: 0.3710, loss: 1.2185 2020-09-06 14:46:26,971 - INFO - Epoch [3][650/1389] lr: 0.01000, eta: 1:25:45, time: 0.378, data_time: 0.004, memory: 9389, loss_cls: 0.8394, loss_bbox: 0.3567, loss: 1.1961 2020-09-06 14:46:46,604 - INFO - Epoch [3][700/1389] lr: 0.01000, eta: 1:25:26, time: 0.393, data_time: 0.004, memory: 9389, loss_cls: 0.8611, loss_bbox: 0.3732, loss: 1.2342 2020-09-06 14:47:06,170 - INFO - Epoch [3][750/1389] lr: 0.01000, eta: 1:25:08, time: 0.391, data_time: 0.004, memory: 9389, loss_cls: 0.8285, loss_bbox: 0.3592, loss: 1.1877 2020-09-06 14:47:24,056 - INFO - Epoch [3][800/1389] lr: 0.01000, eta: 1:24:42, time: 0.358, data_time: 0.003, memory: 9389, loss_cls: 0.9658, loss_bbox: 0.3562, loss: 1.3220 2020-09-06 14:47:43,274 - INFO - Epoch [3][850/1389] lr: 0.01000, eta: 1:24:22, time: 0.384, data_time: 0.004, memory: 9389, loss_cls: 0.9728, loss_bbox: 0.3698, loss: 1.3426 2020-09-06 14:48:03,803 - INFO - Epoch [3][900/1389] lr: 0.01000, eta: 1:24:07, time: 0.411, data_time: 0.004, memory: 9389, loss_cls: 0.8827, loss_bbox: 0.3630, loss: 1.2457 2020-09-06 14:48:23,387 - INFO - Epoch [3][950/1389] lr: 0.01000, eta: 1:23:48, time: 0.392, data_time: 0.004, memory: 9389, loss_cls: 0.8709, loss_bbox: 0.3732, loss: 1.2441 2020-09-06 14:48:42,664 - INFO - Epoch [3][1000/1389] lr: 0.01000, eta: 1:23:28, time: 0.386, data_time: 0.004, memory: 9389, loss_cls: 0.9075, loss_bbox: 0.3688, loss: 1.2763 2020-09-06 14:49:03,580 - INFO - Epoch [3][1050/1389] lr: 0.01000, eta: 1:23:14, time: 0.418, data_time: 0.004, memory: 9389, loss_cls: 0.9000, loss_bbox: 0.3613, loss: 1.2613 2020-09-06 14:49:22,195 - INFO - Epoch [3][1100/1389] lr: 0.01000, eta: 1:22:51, time: 0.372, data_time: 0.004, memory: 9389, loss_cls: 0.8337, loss_bbox: 0.3649, loss: 1.1985 2020-09-06 14:49:42,771 - INFO - Epoch [3][1150/1389] lr: 0.01000, eta: 1:22:36, time: 0.412, data_time: 0.004, memory: 9389, loss_cls: 0.8519, loss_bbox: 0.3605, loss: 1.2124 2020-09-06 14:50:02,409 - INFO - Epoch [3][1200/1389] lr: 0.01000, eta: 1:22:17, time: 0.393, data_time: 0.004, memory: 9389, loss_cls: 0.8386, loss_bbox: 0.3742, loss: 1.2128 2020-09-06 14:50:22,125 - INFO - Epoch [3][1250/1389] lr: 0.01000, eta: 1:21:58, time: 0.394, data_time: 0.004, memory: 9389, loss_cls: 0.8429, loss_bbox: 0.3587, loss: 1.2016 2020-09-06 14:50:42,939 - INFO - Epoch [3][1300/1389] lr: 0.01000, eta: 1:21:43, time: 0.416, data_time: 0.004, memory: 9389, loss_cls: 0.8276, loss_bbox: 0.3735, loss: 1.2011 2020-09-06 14:51:01,350 - INFO - Epoch [3][1350/1389] lr: 0.01000, eta: 1:21:20, time: 0.368, data_time: 0.004, memory: 9389, loss_cls: 0.8277, loss_bbox: 0.3661, loss: 1.1939 2020-09-06 14:51:37,436 - INFO - Epoch [4][50/1389] lr: 0.01000, eta: 1:20:05, time: 0.417, data_time: 0.019, memory: 9389, loss_cls: 0.9050, loss_bbox: 0.3788, loss: 1.2838 2020-09-06 14:51:57,574 - INFO - Epoch [4][100/1389] lr: 0.01000, eta: 1:19:48, time: 0.403, data_time: 0.004, memory: 9389, loss_cls: 0.8515, loss_bbox: 0.3669, loss: 1.2184 2020-09-06 14:52:17,861 - INFO - Epoch [4][150/1389] lr: 0.01000, eta: 1:19:31, time: 0.406, data_time: 0.003, memory: 9389, loss_cls: 0.9160, loss_bbox: 0.3641, loss: 1.2800 2020-09-06 14:52:37,068 - INFO - Epoch [4][200/1389] lr: 0.01000, eta: 1:19:12, time: 0.384, data_time: 0.003, memory: 9389, loss_cls: 0.8346, loss_bbox: 0.3683, loss: 1.2028 2020-09-06 14:52:56,806 - INFO - Epoch [4][250/1389] lr: 0.01000, eta: 1:18:54, time: 0.395, data_time: 0.003, memory: 9389, loss_cls: 0.8182, loss_bbox: 0.3604, loss: 1.1786 2020-09-06 14:53:15,997 - INFO - Epoch [4][300/1389] lr: 0.01000, eta: 1:18:34, time: 0.384, data_time: 0.003, memory: 9389, loss_cls: 0.8454, loss_bbox: 0.3532, loss: 1.1986 2020-09-06 14:53:35,896 - INFO - Epoch [4][350/1389] lr: 0.01000, eta: 1:18:16, time: 0.398, data_time: 0.004, memory: 9389, loss_cls: 0.8419, loss_bbox: 0.3558, loss: 1.1977 2020-09-06 14:53:56,094 - INFO - Epoch [4][400/1389] lr: 0.01000, eta: 1:17:59, time: 0.404, data_time: 0.004, memory: 9389, loss_cls: 0.8323, loss_bbox: 0.3657, loss: 1.1980 2020-09-06 14:54:15,865 - INFO - Epoch [4][450/1389] lr: 0.01000, eta: 1:17:41, time: 0.395, data_time: 0.004, memory: 9389, loss_cls: 0.8820, loss_bbox: 0.3783, loss: 1.2603 2020-09-06 14:54:33,746 - INFO - Epoch [4][500/1389] lr: 0.01000, eta: 1:17:18, time: 0.358, data_time: 0.003, memory: 9389, loss_cls: 0.8850, loss_bbox: 0.3878, loss: 1.2728 2020-09-06 14:54:52,998 - INFO - Epoch [4][550/1389] lr: 0.01000, eta: 1:16:58, time: 0.385, data_time: 0.003, memory: 9389, loss_cls: 0.8530, loss_bbox: 0.3719, loss: 1.2249 2020-09-06 14:55:13,873 - INFO - Epoch [4][600/1389] lr: 0.01000, eta: 1:16:43, time: 0.418, data_time: 0.004, memory: 9389, loss_cls: 0.8732, loss_bbox: 0.3920, loss: 1.2652 2020-09-06 14:55:34,799 - INFO - Epoch [4][650/1389] lr: 0.01000, eta: 1:16:27, time: 0.419, data_time: 0.004, memory: 9389, loss_cls: 0.8441, loss_bbox: 0.3690, loss: 1.2131 2020-09-06 14:55:54,606 - INFO - Epoch [4][700/1389] lr: 0.01000, eta: 1:16:09, time: 0.396, data_time: 0.004, memory: 9389, loss_cls: 0.8134, loss_bbox: 0.3694, loss: 1.1827 2020-09-06 14:56:13,982 - INFO - Epoch [4][750/1389] lr: 0.01000, eta: 1:15:50, time: 0.388, data_time: 0.004, memory: 9389, loss_cls: 0.8264, loss_bbox: 0.3617, loss: 1.1881 2020-09-06 14:56:34,223 - INFO - Epoch [4][800/1389] lr: 0.01000, eta: 1:15:33, time: 0.405, data_time: 0.004, memory: 9389, loss_cls: 0.8325, loss_bbox: 0.3733, loss: 1.2058 2020-09-06 14:56:53,313 - INFO - Epoch [4][850/1389] lr: 0.01000, eta: 1:15:13, time: 0.382, data_time: 0.004, memory: 9389, loss_cls: 0.8257, loss_bbox: 0.3675, loss: 1.1931 2020-09-06 14:57:14,414 - INFO - Epoch [4][900/1389] lr: 0.01000, eta: 1:14:57, time: 0.422, data_time: 0.004, memory: 9389, loss_cls: 0.8362, loss_bbox: 0.3505, loss: 1.1867 2020-09-06 14:57:35,099 - INFO - Epoch [4][950/1389] lr: 0.01000, eta: 1:14:41, time: 0.414, data_time: 0.004, memory: 9389, loss_cls: 0.8864, loss_bbox: 0.3530, loss: 1.2395 2020-09-06 14:57:54,427 - INFO - Epoch [4][1000/1389] lr: 0.01000, eta: 1:14:21, time: 0.387, data_time: 0.004, memory: 9389, loss_cls: 0.8390, loss_bbox: 0.3724, loss: 1.2114 2020-09-06 14:58:13,849 - INFO - Epoch [4][1050/1389] lr: 0.01000, eta: 1:14:02, time: 0.388, data_time: 0.004, memory: 9389, loss_cls: 0.8650, loss_bbox: 0.3646, loss: 1.2296 2020-09-06 14:58:33,196 - INFO - Epoch [4][1100/1389] lr: 0.01000, eta: 1:13:42, time: 0.387, data_time: 0.004, memory: 9389, loss_cls: 0.9035, loss_bbox: 0.3726, loss: 1.2762 2020-09-06 14:58:53,147 - INFO - Epoch [4][1150/1389] lr: 0.01000, eta: 1:13:24, time: 0.399, data_time: 0.004, memory: 9389, loss_cls: 0.8677, loss_bbox: 0.3646, loss: 1.2323 2020-09-06 14:59:13,392 - INFO - Epoch [4][1200/1389] lr: 0.01000, eta: 1:13:06, time: 0.405, data_time: 0.003, memory: 9389, loss_cls: 0.8613, loss_bbox: 0.3641, loss: 1.2254 2020-09-06 14:59:33,537 - INFO - Epoch [4][1250/1389] lr: 0.01000, eta: 1:12:49, time: 0.403, data_time: 0.004, memory: 9389, loss_cls: 0.8591, loss_bbox: 0.3602, loss: 1.2193 2020-09-06 14:59:53,051 - INFO - Epoch [4][1300/1389] lr: 0.01000, eta: 1:12:29, time: 0.390, data_time: 0.004, memory: 9389, loss_cls: 0.8841, loss_bbox: 0.3670, loss: 1.2511 2020-09-06 15:00:12,969 - INFO - Epoch [4][1350/1389] lr: 0.01000, eta: 1:12:11, time: 0.398, data_time: 0.004, memory: 9389, loss_cls: 0.8331, loss_bbox: 0.3625, loss: 1.1956 2020-09-06 15:00:48,208 - INFO - Epoch [5][50/1389] lr: 0.01000, eta: 1:11:10, time: 0.421, data_time: 0.012, memory: 9389, loss_cls: 0.8227, loss_bbox: 0.3521, loss: 1.1747 2020-09-06 15:01:07,991 - INFO - Epoch [5][100/1389] lr: 0.01000, eta: 1:10:51, time: 0.396, data_time: 0.004, memory: 9389, loss_cls: 0.8327, loss_bbox: 0.3679, loss: 1.2006 2020-09-06 15:01:29,859 - INFO - Epoch [5][150/1389] lr: 0.01000, eta: 1:10:37, time: 0.437, data_time: 0.004, memory: 9389, loss_cls: 0.8552, loss_bbox: 0.3708, loss: 1.2260 2020-09-06 15:01:49,993 - INFO - Epoch [5][200/1389] lr: 0.01000, eta: 1:10:19, time: 0.403, data_time: 0.004, memory: 9389, loss_cls: 0.8428, loss_bbox: 0.3602, loss: 1.2030 2020-09-06 15:02:08,277 - INFO - Epoch [5][250/1389] lr: 0.01000, eta: 1:09:58, time: 0.366, data_time: 0.003, memory: 9389, loss_cls: 0.8359, loss_bbox: 0.3629, loss: 1.1988 2020-09-06 15:02:28,338 - INFO - Epoch [5][300/1389] lr: 0.01000, eta: 1:09:40, time: 0.401, data_time: 0.004, memory: 9389, loss_cls: 0.8657, loss_bbox: 0.3607, loss: 1.2264 2020-09-06 15:02:49,172 - INFO - Epoch [5][350/1389] lr: 0.01000, eta: 1:09:23, time: 0.417, data_time: 0.004, memory: 9389, loss_cls: 0.8081, loss_bbox: 0.3601, loss: 1.1682 2020-09-06 15:03:09,548 - INFO - Epoch [5][400/1389] lr: 0.01000, eta: 1:09:06, time: 0.408, data_time: 0.004, memory: 9389, loss_cls: 0.8456, loss_bbox: 0.3545, loss: 1.2001 2020-09-06 15:03:28,421 - INFO - Epoch [5][450/1389] lr: 0.01000, eta: 1:08:46, time: 0.377, data_time: 0.004, memory: 9389, loss_cls: 0.8526, loss_bbox: 0.3578, loss: 1.2104 2020-09-06 15:03:48,079 - INFO - Epoch [5][500/1389] lr: 0.01000, eta: 1:08:27, time: 0.393, data_time: 0.004, memory: 9389, loss_cls: 0.7944, loss_bbox: 0.3501, loss: 1.1445 2020-09-06 15:04:07,202 - INFO - Epoch [5][550/1389] lr: 0.01000, eta: 1:08:07, time: 0.382, data_time: 0.004, memory: 9389, loss_cls: 1.0631, loss_bbox: 0.3645, loss: 1.4275 2020-09-06 15:04:26,216 - INFO - Epoch [5][600/1389] lr: 0.01000, eta: 1:07:47, time: 0.380, data_time: 0.003, memory: 9389, loss_cls: 1.0379, loss_bbox: 0.3785, loss: 1.4164 2020-09-06 15:04:44,371 - INFO - Epoch [5][650/1389] lr: 0.01000, eta: 1:07:26, time: 0.363, data_time: 0.003, memory: 9389, loss_cls: 0.8357, loss_bbox: 0.3537, loss: 1.1894 2020-09-06 15:05:04,796 - INFO - Epoch [5][700/1389] lr: 0.01000, eta: 1:07:08, time: 0.408, data_time: 0.004, memory: 9389, loss_cls: 0.8503, loss_bbox: 0.3741, loss: 1.2244 2020-09-06 15:05:23,951 - INFO - Epoch [5][750/1389] lr: 0.01000, eta: 1:06:49, time: 0.383, data_time: 0.004, memory: 9389, loss_cls: 0.8441, loss_bbox: 0.3705, loss: 1.2147 2020-09-06 15:05:42,665 - INFO - Epoch [5][800/1389] lr: 0.01000, eta: 1:06:28, time: 0.374, data_time: 0.003, memory: 9389, loss_cls: 0.8585, loss_bbox: 0.3725, loss: 1.2310 2020-09-06 15:06:03,521 - INFO - Epoch [5][850/1389] lr: 0.01000, eta: 1:06:11, time: 0.417, data_time: 0.004, memory: 9389, loss_cls: 0.8157, loss_bbox: 0.3688, loss: 1.1845 2020-09-06 15:06:22,821 - INFO - Epoch [5][900/1389] lr: 0.01000, eta: 1:05:52, time: 0.386, data_time: 0.003, memory: 9389, loss_cls: 0.8295, loss_bbox: 0.3630, loss: 1.1925 2020-09-06 15:06:42,412 - INFO - Epoch [5][950/1389] lr: 0.01000, eta: 1:05:33, time: 0.392, data_time: 0.004, memory: 9389, loss_cls: 0.8676, loss_bbox: 0.3690, loss: 1.2365 2020-09-06 15:07:01,413 - INFO - Epoch [5][1000/1389] lr: 0.01000, eta: 1:05:13, time: 0.380, data_time: 0.004, memory: 9389, loss_cls: 0.8496, loss_bbox: 0.3682, loss: 1.2178 2020-09-06 15:07:20,679 - INFO - Epoch [5][1050/1389] lr: 0.01000, eta: 1:04:53, time: 0.385, data_time: 0.004, memory: 9389, loss_cls: 0.8677, loss_bbox: 0.3619, loss: 1.2296 2020-09-06 15:07:41,413 - INFO - Epoch [5][1100/1389] lr: 0.01000, eta: 1:04:36, time: 0.415, data_time: 0.004, memory: 9389, loss_cls: 0.8640, loss_bbox: 0.3807, loss: 1.2447 2020-09-06 15:08:00,685 - INFO - Epoch [5][1150/1389] lr: 0.01000, eta: 1:04:17, time: 0.385, data_time: 0.004, memory: 9389, loss_cls: 0.9415, loss_bbox: 0.3566, loss: 1.2982 2020-09-06 15:08:19,968 - INFO - Epoch [5][1200/1389] lr: 0.01000, eta: 1:03:57, time: 0.386, data_time: 0.004, memory: 9389, loss_cls: 0.9265, loss_bbox: 0.3666, loss: 1.2930 2020-09-06 15:08:38,397 - INFO - Epoch [5][1250/1389] lr: 0.01000, eta: 1:03:37, time: 0.369, data_time: 0.004, memory: 9389, loss_cls: 0.8456, loss_bbox: 0.3608, loss: 1.2064 2020-09-06 15:08:57,745 - INFO - Epoch [5][1300/1389] lr: 0.01000, eta: 1:03:17, time: 0.387, data_time: 0.004, memory: 9389, loss_cls: 0.8577, loss_bbox: 0.3692, loss: 1.2269 2020-09-06 15:09:18,499 - INFO - Epoch [5][1350/1389] lr: 0.01000, eta: 1:03:00, time: 0.415, data_time: 0.004, memory: 9389, loss_cls: 0.8742, loss_bbox: 0.3650, loss: 1.2392 2020-09-06 15:09:53,858 - INFO - Epoch [6][50/1389] lr: 0.01000, eta: 1:02:05, time: 0.390, data_time: 0.013, memory: 9389, loss_cls: 0.8329, loss_bbox: 0.3552, loss: 1.1881 2020-09-06 15:10:14,346 - INFO - Epoch [6][100/1389] lr: 0.01000, eta: 1:01:47, time: 0.410, data_time: 0.004, memory: 9389, loss_cls: 0.8936, loss_bbox: 0.3601, loss: 1.2537 2020-09-06 15:10:35,096 - INFO - Epoch [6][150/1389] lr: 0.01000, eta: 1:01:30, time: 0.415, data_time: 0.004, memory: 9389, loss_cls: 0.8422, loss_bbox: 0.3598, loss: 1.2020 2020-09-06 15:10:54,617 - INFO - Epoch [6][200/1389] lr: 0.01000, eta: 1:01:11, time: 0.390, data_time: 0.003, memory: 9389, loss_cls: 0.8535, loss_bbox: 0.3622, loss: 1.2158 2020-09-06 15:11:14,662 - INFO - Epoch [6][250/1389] lr: 0.01000, eta: 1:00:53, time: 0.401, data_time: 0.004, memory: 9389, loss_cls: 0.9344, loss_bbox: 0.3749, loss: 1.3093 2020-09-06 15:11:33,514 - INFO - Epoch [6][300/1389] lr: 0.01000, eta: 1:00:33, time: 0.377, data_time: 0.003, memory: 9389, loss_cls: 0.8928, loss_bbox: 0.3683, loss: 1.2611 2020-09-06 15:11:53,061 - INFO - Epoch [6][350/1389] lr: 0.01000, eta: 1:00:14, time: 0.391, data_time: 0.003, memory: 9389, loss_cls: 0.8525, loss_bbox: 0.3653, loss: 1.2178 2020-09-06 15:12:13,225 - INFO - Epoch [6][400/1389] lr: 0.01000, eta: 0:59:56, time: 0.403, data_time: 0.004, memory: 9389, loss_cls: 0.8901, loss_bbox: 0.3673, loss: 1.2574 2020-09-06 15:12:31,976 - INFO - Epoch [6][450/1389] lr: 0.01000, eta: 0:59:36, time: 0.375, data_time: 0.004, memory: 9389, loss_cls: 0.8424, loss_bbox: 0.3696, loss: 1.2121 2020-09-06 15:12:50,798 - INFO - Epoch [6][500/1389] lr: 0.01000, eta: 0:59:16, time: 0.376, data_time: 0.003, memory: 9389, loss_cls: 0.8378, loss_bbox: 0.3704, loss: 1.2082 2020-09-06 15:13:10,348 - INFO - Epoch [6][550/1389] lr: 0.01000, eta: 0:58:57, time: 0.391, data_time: 0.003, memory: 9389, loss_cls: 0.8376, loss_bbox: 0.3621, loss: 1.1997 2020-09-06 15:13:30,220 - INFO - Epoch [6][600/1389] lr: 0.01000, eta: 0:58:38, time: 0.397, data_time: 0.004, memory: 9389, loss_cls: 0.8426, loss_bbox: 0.3662, loss: 1.2088 2020-09-06 15:13:49,726 - INFO - Epoch [6][650/1389] lr: 0.01000, eta: 0:58:19, time: 0.390, data_time: 0.004, memory: 9389, loss_cls: 0.8544, loss_bbox: 0.3731, loss: 1.2275 2020-09-06 15:14:07,878 - INFO - Epoch [6][700/1389] lr: 0.01000, eta: 0:57:59, time: 0.363, data_time: 0.003, memory: 9389, loss_cls: 0.9088, loss_bbox: 0.3578, loss: 1.2667 2020-09-06 15:14:28,793 - INFO - Epoch [6][750/1389] lr: 0.01000, eta: 0:57:41, time: 0.418, data_time: 0.004, memory: 9389, loss_cls: 0.9157, loss_bbox: 0.3538, loss: 1.2694 2020-09-06 15:14:48,962 - INFO - Epoch [6][800/1389] lr: 0.01000, eta: 0:57:23, time: 0.403, data_time: 0.004, memory: 9389, loss_cls: 0.8653, loss_bbox: 0.3638, loss: 1.2291 2020-09-06 15:15:08,412 - INFO - Epoch [6][850/1389] lr: 0.01000, eta: 0:57:04, time: 0.389, data_time: 0.004, memory: 9389, loss_cls: 0.8281, loss_bbox: 0.3628, loss: 1.1909 2020-09-06 15:15:27,884 - INFO - Epoch [6][900/1389] lr: 0.01000, eta: 0:56:45, time: 0.389, data_time: 0.004, memory: 9389, loss_cls: 0.8319, loss_bbox: 0.3560, loss: 1.1879 2020-09-06 15:15:46,060 - INFO - Epoch [6][950/1389] lr: 0.01000, eta: 0:56:24, time: 0.364, data_time: 0.003, memory: 9389, loss_cls: 1.0060, loss_bbox: 0.3850, loss: 1.3909 2020-09-06 15:16:05,235 - INFO - Epoch [6][1000/1389] lr: 0.01000, eta: 0:56:05, time: 0.383, data_time: 0.003, memory: 9389, loss_cls: 0.8386, loss_bbox: 0.3657, loss: 1.2043 2020-09-06 15:16:25,394 - INFO - Epoch [6][1050/1389] lr: 0.01000, eta: 0:55:47, time: 0.403, data_time: 0.004, memory: 9389, loss_cls: 0.8431, loss_bbox: 0.3643, loss: 1.2073 2020-09-06 15:16:44,221 - INFO - Epoch [6][1100/1389] lr: 0.01000, eta: 0:55:27, time: 0.377, data_time: 0.004, memory: 9389, loss_cls: 0.8389, loss_bbox: 0.3509, loss: 1.1899 2020-09-06 15:17:03,383 - INFO - Epoch [6][1150/1389] lr: 0.01000, eta: 0:55:07, time: 0.383, data_time: 0.004, memory: 9389, loss_cls: 0.8502, loss_bbox: 0.3569, loss: 1.2071 2020-09-06 15:17:23,081 - INFO - Epoch [6][1200/1389] lr: 0.01000, eta: 0:54:48, time: 0.394, data_time: 0.004, memory: 9389, loss_cls: 0.8920, loss_bbox: 0.3648, loss: 1.2569 2020-09-06 15:17:42,522 - INFO - Epoch [6][1250/1389] lr: 0.01000, eta: 0:54:29, time: 0.389, data_time: 0.004, memory: 9389, loss_cls: 0.8235, loss_bbox: 0.3624, loss: 1.1859 2020-09-06 15:18:01,933 - INFO - Epoch [6][1300/1389] lr: 0.01000, eta: 0:54:10, time: 0.388, data_time: 0.004, memory: 9389, loss_cls: 0.8313, loss_bbox: 0.3566, loss: 1.1879 2020-09-06 15:18:21,351 - INFO - Epoch [6][1350/1389] lr: 0.01000, eta: 0:53:51, time: 0.388, data_time: 0.004, memory: 9389, loss_cls: 0.8294, loss_bbox: 0.3649, loss: 1.1943 2020-09-06 15:18:58,822 - INFO - Epoch [7][50/1389] lr: 0.01000, eta: 0:53:04, time: 0.425, data_time: 0.013, memory: 9389, loss_cls: 0.8741, loss_bbox: 0.3566, loss: 1.2307 2020-09-06 15:19:20,225 - INFO - Epoch [7][100/1389] lr: 0.01000, eta: 0:52:47, time: 0.428, data_time: 0.004, memory: 9389, loss_cls: 0.8135, loss_bbox: 0.3540, loss: 1.1674 2020-09-06 15:19:40,562 - INFO - Epoch [7][150/1389] lr: 0.01000, eta: 0:52:28, time: 0.407, data_time: 0.004, memory: 9389, loss_cls: 0.9036, loss_bbox: 0.3673, loss: 1.2710 2020-09-06 15:20:00,774 - INFO - Epoch [7][200/1389] lr: 0.01000, eta: 0:52:10, time: 0.404, data_time: 0.003, memory: 9389, loss_cls: 0.8279, loss_bbox: 0.3674, loss: 1.1952 2020-09-06 15:20:20,088 - INFO - Epoch [7][250/1389] lr: 0.01000, eta: 0:51:51, time: 0.386, data_time: 0.004, memory: 9389, loss_cls: 0.9090, loss_bbox: 0.3634, loss: 1.2724 2020-09-06 15:20:38,736 - INFO - Epoch [7][300/1389] lr: 0.01000, eta: 0:51:31, time: 0.373, data_time: 0.003, memory: 9389, loss_cls: 0.8581, loss_bbox: 0.3635, loss: 1.2216 2020-09-06 15:20:58,699 - INFO - Epoch [7][350/1389] lr: 0.01000, eta: 0:51:13, time: 0.399, data_time: 0.004, memory: 9389, loss_cls: 0.8295, loss_bbox: 0.3725, loss: 1.2020 2020-09-06 15:21:18,073 - INFO - Epoch [7][400/1389] lr: 0.01000, eta: 0:50:53, time: 0.387, data_time: 0.003, memory: 9389, loss_cls: 0.8466, loss_bbox: 0.3671, loss: 1.2137 2020-09-06 15:21:37,168 - INFO - Epoch [7][450/1389] lr: 0.01000, eta: 0:50:34, time: 0.382, data_time: 0.003, memory: 9389, loss_cls: 0.8190, loss_bbox: 0.3508, loss: 1.1698 2020-09-06 15:21:56,005 - INFO - Epoch [7][500/1389] lr: 0.01000, eta: 0:50:14, time: 0.377, data_time: 0.004, memory: 9389, loss_cls: 0.8602, loss_bbox: 0.3600, loss: 1.2203 2020-09-06 15:22:16,328 - INFO - Epoch [7][550/1389] lr: 0.01000, eta: 0:49:56, time: 0.406, data_time: 0.004, memory: 9389, loss_cls: 0.8401, loss_bbox: 0.3592, loss: 1.1993 2020-09-06 15:22:36,531 - INFO - Epoch [7][600/1389] lr: 0.01000, eta: 0:49:38, time: 0.404, data_time: 0.004, memory: 9389, loss_cls: 0.8350, loss_bbox: 0.3631, loss: 1.1981 2020-09-06 15:22:56,581 - INFO - Epoch [7][650/1389] lr: 0.01000, eta: 0:49:19, time: 0.401, data_time: 0.004, memory: 9389, loss_cls: 0.8524, loss_bbox: 0.3545, loss: 1.2069 2020-09-06 15:23:16,427 - INFO - Epoch [7][700/1389] lr: 0.01000, eta: 0:49:00, time: 0.397, data_time: 0.004, memory: 9389, loss_cls: 0.8767, loss_bbox: 0.3823, loss: 1.2590 2020-09-06 15:23:35,978 - INFO - Epoch [7][750/1389] lr: 0.01000, eta: 0:48:41, time: 0.391, data_time: 0.004, memory: 9389, loss_cls: 0.8158, loss_bbox: 0.3591, loss: 1.1749 2020-09-06 15:23:55,761 - INFO - Epoch [7][800/1389] lr: 0.01000, eta: 0:48:23, time: 0.396, data_time: 0.004, memory: 9389, loss_cls: 0.8446, loss_bbox: 0.3638, loss: 1.2084 2020-09-06 15:24:15,958 - INFO - Epoch [7][850/1389] lr: 0.01000, eta: 0:48:04, time: 0.404, data_time: 0.004, memory: 9389, loss_cls: 0.8823, loss_bbox: 0.3526, loss: 1.2349 2020-09-06 15:24:36,231 - INFO - Epoch [7][900/1389] lr: 0.01000, eta: 0:47:46, time: 0.405, data_time: 0.004, memory: 9389, loss_cls: 0.8906, loss_bbox: 0.3587, loss: 1.2493 2020-09-06 15:24:56,448 - INFO - Epoch [7][950/1389] lr: 0.01000, eta: 0:47:27, time: 0.404, data_time: 0.004, memory: 9389, loss_cls: 0.8866, loss_bbox: 0.3603, loss: 1.2469 2020-09-06 15:25:15,491 - INFO - Epoch [7][1000/1389] lr: 0.01000, eta: 0:47:08, time: 0.381, data_time: 0.004, memory: 9389, loss_cls: 0.8351, loss_bbox: 0.3691, loss: 1.2041 2020-09-06 15:25:34,828 - INFO - Epoch [7][1050/1389] lr: 0.01000, eta: 0:46:48, time: 0.387, data_time: 0.004, memory: 9389, loss_cls: 0.8597, loss_bbox: 0.3591, loss: 1.2189 2020-09-06 15:25:55,557 - INFO - Epoch [7][1100/1389] lr: 0.01000, eta: 0:46:30, time: 0.415, data_time: 0.004, memory: 9389, loss_cls: 0.8398, loss_bbox: 0.3684, loss: 1.2082 2020-09-06 15:26:15,502 - INFO - Epoch [7][1150/1389] lr: 0.01000, eta: 0:46:11, time: 0.399, data_time: 0.004, memory: 9389, loss_cls: 0.8180, loss_bbox: 0.3561, loss: 1.1741 2020-09-06 15:26:34,456 - INFO - Epoch [7][1200/1389] lr: 0.01000, eta: 0:45:52, time: 0.379, data_time: 0.004, memory: 9389, loss_cls: 0.8215, loss_bbox: 0.3584, loss: 1.1798 2020-09-06 15:26:55,054 - INFO - Epoch [7][1250/1389] lr: 0.01000, eta: 0:45:33, time: 0.412, data_time: 0.004, memory: 9389, loss_cls: 0.8605, loss_bbox: 0.3484, loss: 1.2089 2020-09-06 15:27:13,467 - INFO - Epoch [7][1300/1389] lr: 0.01000, eta: 0:45:14, time: 0.368, data_time: 0.004, memory: 9389, loss_cls: 0.8710, loss_bbox: 0.3771, loss: 1.2481 2020-09-06 15:27:33,037 - INFO - Epoch [7][1350/1389] lr: 0.01000, eta: 0:44:54, time: 0.391, data_time: 0.004, memory: 9389, loss_cls: 0.8343, loss_bbox: 0.3703, loss: 1.2046 2020-09-06 15:28:07,913 - INFO - Epoch [8][50/1389] lr: 0.01000, eta: 0:44:10, time: 0.387, data_time: 0.012, memory: 9389, loss_cls: 0.9087, loss_bbox: 0.3605, loss: 1.2691 2020-09-06 15:28:29,178 - INFO - Epoch [8][100/1389] lr: 0.01000, eta: 0:43:52, time: 0.425, data_time: 0.004, memory: 9389, loss_cls: 0.8249, loss_bbox: 0.3714, loss: 1.1963 2020-09-06 15:28:49,193 - INFO - Epoch [8][150/1389] lr: 0.01000, eta: 0:43:33, time: 0.400, data_time: 0.004, memory: 9389, loss_cls: 0.8689, loss_bbox: 0.3612, loss: 1.2301 2020-09-06 15:29:09,447 - INFO - Epoch [8][200/1389] lr: 0.01000, eta: 0:43:15, time: 0.405, data_time: 0.004, memory: 9389, loss_cls: 0.8645, loss_bbox: 0.3710, loss: 1.2356 2020-09-06 15:29:29,106 - INFO - Epoch [8][250/1389] lr: 0.01000, eta: 0:42:56, time: 0.393, data_time: 0.004, memory: 9389, loss_cls: 0.8673, loss_bbox: 0.3591, loss: 1.2264 2020-09-06 15:29:48,004 - INFO - Epoch [8][300/1389] lr: 0.01000, eta: 0:42:36, time: 0.378, data_time: 0.003, memory: 9389, loss_cls: 0.8674, loss_bbox: 0.3507, loss: 1.2181 2020-09-06 15:30:07,492 - INFO - Epoch [8][350/1389] lr: 0.01000, eta: 0:42:17, time: 0.390, data_time: 0.004, memory: 9389, loss_cls: 0.9612, loss_bbox: 0.3752, loss: 1.3364 2020-09-06 15:30:27,740 - INFO - Epoch [8][400/1389] lr: 0.01000, eta: 0:41:58, time: 0.405, data_time: 0.004, memory: 9389, loss_cls: 0.8189, loss_bbox: 0.3582, loss: 1.1771 2020-09-06 15:30:46,665 - INFO - Epoch [8][450/1389] lr: 0.01000, eta: 0:41:39, time: 0.379, data_time: 0.003, memory: 9389, loss_cls: 0.8734, loss_bbox: 0.3640, loss: 1.2374 2020-09-06 15:31:05,171 - INFO - Epoch [8][500/1389] lr: 0.01000, eta: 0:41:19, time: 0.370, data_time: 0.004, memory: 9389, loss_cls: 0.8410, loss_bbox: 0.3750, loss: 1.2160 2020-09-06 15:31:24,130 - INFO - Epoch [8][550/1389] lr: 0.01000, eta: 0:41:00, time: 0.379, data_time: 0.004, memory: 9389, loss_cls: 0.8473, loss_bbox: 0.3542, loss: 1.2015 2020-09-06 15:31:44,431 - INFO - Epoch [8][600/1389] lr: 0.01000, eta: 0:40:41, time: 0.406, data_time: 0.004, memory: 9389, loss_cls: 0.8981, loss_bbox: 0.3803, loss: 1.2784 2020-09-06 15:32:03,266 - INFO - Epoch [8][650/1389] lr: 0.01000, eta: 0:40:22, time: 0.377, data_time: 0.004, memory: 9389, loss_cls: 0.9056, loss_bbox: 0.3762, loss: 1.2818 2020-09-06 15:32:23,314 - INFO - Epoch [8][700/1389] lr: 0.01000, eta: 0:40:03, time: 0.401, data_time: 0.004, memory: 9389, loss_cls: 0.8844, loss_bbox: 0.3658, loss: 1.2502 2020-09-06 15:32:42,233 - INFO - Epoch [8][750/1389] lr: 0.01000, eta: 0:39:44, time: 0.378, data_time: 0.004, memory: 9389, loss_cls: 0.8330, loss_bbox: 0.3678, loss: 1.2008 2020-09-06 15:33:02,267 - INFO - Epoch [8][800/1389] lr: 0.01000, eta: 0:39:25, time: 0.401, data_time: 0.004, memory: 9389, loss_cls: 0.8291, loss_bbox: 0.3581, loss: 1.1872 2020-09-06 15:33:20,507 - INFO - Epoch [8][850/1389] lr: 0.01000, eta: 0:39:05, time: 0.365, data_time: 0.004, memory: 9389, loss_cls: 0.8494, loss_bbox: 0.3697, loss: 1.2191 2020-09-06 15:33:39,133 - INFO - Epoch [8][900/1389] lr: 0.01000, eta: 0:38:45, time: 0.373, data_time: 0.003, memory: 9389, loss_cls: 0.8379, loss_bbox: 0.3641, loss: 1.2020 2020-09-06 15:33:58,157 - INFO - Epoch [8][950/1389] lr: 0.01000, eta: 0:38:26, time: 0.380, data_time: 0.004, memory: 9389, loss_cls: 0.8924, loss_bbox: 0.3686, loss: 1.2610 2020-09-06 15:34:18,157 - INFO - Epoch [8][1000/1389] lr: 0.01000, eta: 0:38:07, time: 0.400, data_time: 0.004, memory: 9389, loss_cls: 0.8589, loss_bbox: 0.3549, loss: 1.2138 2020-09-06 15:34:37,992 - INFO - Epoch [8][1050/1389] lr: 0.01000, eta: 0:37:48, time: 0.397, data_time: 0.004, memory: 9389, loss_cls: 0.8494, loss_bbox: 0.3651, loss: 1.2144 2020-09-06 15:34:58,597 - INFO - Epoch [8][1100/1389] lr: 0.01000, eta: 0:37:30, time: 0.412, data_time: 0.004, memory: 9389, loss_cls: 0.8206, loss_bbox: 0.3703, loss: 1.1909 2020-09-06 15:35:18,463 - INFO - Epoch [8][1150/1389] lr: 0.01000, eta: 0:37:11, time: 0.397, data_time: 0.004, memory: 9389, loss_cls: 0.9676, loss_bbox: 0.3735, loss: 1.3411 2020-09-06 15:35:38,398 - INFO - Epoch [8][1200/1389] lr: 0.01000, eta: 0:36:52, time: 0.399, data_time: 0.004, memory: 9389, loss_cls: 1.1434, loss_bbox: 0.3739, loss: 1.5173 2020-09-06 15:35:57,089 - INFO - Epoch [8][1250/1389] lr: 0.01000, eta: 0:36:32, time: 0.374, data_time: 0.003, memory: 9389, loss_cls: 0.9001, loss_bbox: 0.3819, loss: 1.2819 2020-09-06 15:36:17,351 - INFO - Epoch [8][1300/1389] lr: 0.01000, eta: 0:36:14, time: 0.405, data_time: 0.004, memory: 9389, loss_cls: 0.8347, loss_bbox: 0.3579, loss: 1.1927 2020-09-06 15:36:37,398 - INFO - Epoch [8][1350/1389] lr: 0.01000, eta: 0:35:55, time: 0.401, data_time: 0.004, memory: 9389, loss_cls: 0.8468, loss_bbox: 0.3570, loss: 1.2038 2020-09-06 15:37:16,713 - INFO - Epoch [9][50/1389] lr: 0.00100, eta: 0:35:14, time: 0.439, data_time: 0.015, memory: 9389, loss_cls: 0.8816, loss_bbox: 0.3614, loss: 1.2430 2020-09-06 15:37:37,385 - INFO - Epoch [9][100/1389] lr: 0.00100, eta: 0:34:56, time: 0.413, data_time: 0.004, memory: 9389, loss_cls: 0.8340, loss_bbox: 0.3632, loss: 1.1972 2020-09-06 15:37:57,825 - INFO - Epoch [9][150/1389] lr: 0.00100, eta: 0:34:37, time: 0.409, data_time: 0.004, memory: 9389, loss_cls: 0.8478, loss_bbox: 0.3560, loss: 1.2038 2020-09-06 15:38:17,890 - INFO - Epoch [9][200/1389] lr: 0.00100, eta: 0:34:19, time: 0.401, data_time: 0.004, memory: 9389, loss_cls: 0.7991, loss_bbox: 0.3488, loss: 1.1479 2020-09-06 15:38:36,807 - INFO - Epoch [9][250/1389] lr: 0.00100, eta: 0:33:59, time: 0.378, data_time: 0.003, memory: 9389, loss_cls: 0.8636, loss_bbox: 0.3803, loss: 1.2439 2020-09-06 15:38:57,529 - INFO - Epoch [9][300/1389] lr: 0.00100, eta: 0:33:41, time: 0.414, data_time: 0.004, memory: 9389, loss_cls: 0.8335, loss_bbox: 0.3594, loss: 1.1929 2020-09-06 15:39:16,846 - INFO - Epoch [9][350/1389] lr: 0.00100, eta: 0:33:21, time: 0.386, data_time: 0.003, memory: 9389, loss_cls: 0.8282, loss_bbox: 0.3600, loss: 1.1882 2020-09-06 15:39:35,391 - INFO - Epoch [9][400/1389] lr: 0.00100, eta: 0:33:02, time: 0.371, data_time: 0.003, memory: 9389, loss_cls: 0.8091, loss_bbox: 0.3592, loss: 1.1683 2020-09-06 15:39:54,757 - INFO - Epoch [9][450/1389] lr: 0.00100, eta: 0:32:43, time: 0.387, data_time: 0.003, memory: 9389, loss_cls: 0.8398, loss_bbox: 0.3555, loss: 1.1953 2020-09-06 15:40:14,207 - INFO - Epoch [9][500/1389] lr: 0.00100, eta: 0:32:24, time: 0.389, data_time: 0.004, memory: 9389, loss_cls: 0.8293, loss_bbox: 0.3538, loss: 1.1831 2020-09-06 15:40:35,334 - INFO - Epoch [9][550/1389] lr: 0.00100, eta: 0:32:05, time: 0.423, data_time: 0.004, memory: 9389, loss_cls: 0.8061, loss_bbox: 0.3501, loss: 1.1562 2020-09-06 15:40:55,505 - INFO - Epoch [9][600/1389] lr: 0.00100, eta: 0:31:46, time: 0.403, data_time: 0.004, memory: 9389, loss_cls: 0.8518, loss_bbox: 0.3602, loss: 1.2120 2020-09-06 15:41:13,099 - INFO - Epoch [9][650/1389] lr: 0.00100, eta: 0:31:26, time: 0.352, data_time: 0.004, memory: 9389, loss_cls: 0.8580, loss_bbox: 0.3592, loss: 1.2172 2020-09-06 15:41:32,945 - INFO - Epoch [9][700/1389] lr: 0.00100, eta: 0:31:07, time: 0.397, data_time: 0.004, memory: 9389, loss_cls: 0.8278, loss_bbox: 0.3589, loss: 1.1867 2020-09-06 15:41:53,252 - INFO - Epoch [9][750/1389] lr: 0.00100, eta: 0:30:49, time: 0.406, data_time: 0.004, memory: 9389, loss_cls: 0.8772, loss_bbox: 0.3646, loss: 1.2418 2020-09-06 15:42:14,354 - INFO - Epoch [9][800/1389] lr: 0.00100, eta: 0:30:30, time: 0.422, data_time: 0.004, memory: 9389, loss_cls: 0.8729, loss_bbox: 0.3587, loss: 1.2316 2020-09-06 15:42:34,156 - INFO - Epoch [9][850/1389] lr: 0.00100, eta: 0:30:11, time: 0.396, data_time: 0.004, memory: 9389, loss_cls: 0.8379, loss_bbox: 0.3548, loss: 1.1927 2020-09-06 15:42:52,845 - INFO - Epoch [9][900/1389] lr: 0.00100, eta: 0:29:52, time: 0.374, data_time: 0.003, memory: 9389, loss_cls: 0.8079, loss_bbox: 0.3561, loss: 1.1640 2020-09-06 15:43:12,358 - INFO - Epoch [9][950/1389] lr: 0.00100, eta: 0:29:33, time: 0.390, data_time: 0.004, memory: 9389, loss_cls: 0.8640, loss_bbox: 0.3723, loss: 1.2363 2020-09-06 15:43:32,466 - INFO - Epoch [9][1000/1389] lr: 0.00100, eta: 0:29:14, time: 0.402, data_time: 0.004, memory: 9389, loss_cls: 0.8123, loss_bbox: 0.3619, loss: 1.1742 2020-09-06 15:43:53,930 - INFO - Epoch [9][1050/1389] lr: 0.00100, eta: 0:28:55, time: 0.429, data_time: 0.004, memory: 9389, loss_cls: 0.8509, loss_bbox: 0.3691, loss: 1.2200 2020-09-06 15:44:13,693 - INFO - Epoch [9][1100/1389] lr: 0.00100, eta: 0:28:36, time: 0.395, data_time: 0.004, memory: 9389, loss_cls: 0.8360, loss_bbox: 0.3522, loss: 1.1882 2020-09-06 15:44:33,824 - INFO - Epoch [9][1150/1389] lr: 0.00100, eta: 0:28:17, time: 0.403, data_time: 0.004, memory: 9389, loss_cls: 0.8326, loss_bbox: 0.3642, loss: 1.1967 2020-09-06 15:44:53,900 - INFO - Epoch [9][1200/1389] lr: 0.00100, eta: 0:27:58, time: 0.402, data_time: 0.004, memory: 9389, loss_cls: 0.8340, loss_bbox: 0.3656, loss: 1.1996 2020-09-06 15:45:13,553 - INFO - Epoch [9][1250/1389] lr: 0.00100, eta: 0:27:39, time: 0.393, data_time: 0.004, memory: 9389, loss_cls: 0.8286, loss_bbox: 0.3656, loss: 1.1941 2020-09-06 15:45:32,710 - INFO - Epoch [9][1300/1389] lr: 0.00100, eta: 0:27:20, time: 0.383, data_time: 0.004, memory: 9389, loss_cls: 0.8213, loss_bbox: 0.3543, loss: 1.1756 2020-09-06 15:45:52,543 - INFO - Epoch [9][1350/1389] lr: 0.00100, eta: 0:27:01, time: 0.397, data_time: 0.004, memory: 9389, loss_cls: 0.8226, loss_bbox: 0.3551, loss: 1.1777 2020-09-06 15:46:29,644 - INFO - Epoch [10][50/1389] lr: 0.00100, eta: 0:26:22, time: 0.430, data_time: 0.016, memory: 9389, loss_cls: 0.8495, loss_bbox: 0.3500, loss: 1.1995 2020-09-06 15:46:48,469 - INFO - Epoch [10][100/1389] lr: 0.00100, eta: 0:26:03, time: 0.376, data_time: 0.004, memory: 9389, loss_cls: 0.8399, loss_bbox: 0.3762, loss: 1.2161 2020-09-06 15:47:08,200 - INFO - Epoch [10][150/1389] lr: 0.00100, eta: 0:25:44, time: 0.395, data_time: 0.004, memory: 9389, loss_cls: 0.8255, loss_bbox: 0.3694, loss: 1.1949 2020-09-06 15:47:28,314 - INFO - Epoch [10][200/1389] lr: 0.00100, eta: 0:25:25, time: 0.402, data_time: 0.004, memory: 9389, loss_cls: 0.8090, loss_bbox: 0.3544, loss: 1.1634 2020-09-06 15:47:46,902 - INFO - Epoch [10][250/1389] lr: 0.00100, eta: 0:25:05, time: 0.372, data_time: 0.004, memory: 9389, loss_cls: 0.8433, loss_bbox: 0.3569, loss: 1.2001 2020-09-06 15:48:05,254 - INFO - Epoch [10][300/1389] lr: 0.00100, eta: 0:24:46, time: 0.367, data_time: 0.004, memory: 9389, loss_cls: 0.8283, loss_bbox: 0.3655, loss: 1.1938 2020-09-06 15:48:24,798 - INFO - Epoch [10][350/1389] lr: 0.00100, eta: 0:24:27, time: 0.391, data_time: 0.004, memory: 9389, loss_cls: 0.8144, loss_bbox: 0.3564, loss: 1.1708 2020-09-06 15:48:44,382 - INFO - Epoch [10][400/1389] lr: 0.00100, eta: 0:24:08, time: 0.392, data_time: 0.004, memory: 9389, loss_cls: 0.8135, loss_bbox: 0.3555, loss: 1.1690 2020-09-06 15:49:04,546 - INFO - Epoch [10][450/1389] lr: 0.00100, eta: 0:23:49, time: 0.403, data_time: 0.004, memory: 9389, loss_cls: 0.8347, loss_bbox: 0.3574, loss: 1.1921 2020-09-06 15:49:23,770 - INFO - Epoch [10][500/1389] lr: 0.00100, eta: 0:23:29, time: 0.384, data_time: 0.004, memory: 9389, loss_cls: 0.8643, loss_bbox: 0.3600, loss: 1.2244 2020-09-06 15:49:44,127 - INFO - Epoch [10][550/1389] lr: 0.00100, eta: 0:23:11, time: 0.407, data_time: 0.004, memory: 9389, loss_cls: 0.8066, loss_bbox: 0.3610, loss: 1.1676 2020-09-06 15:50:04,606 - INFO - Epoch [10][600/1389] lr: 0.00100, eta: 0:22:52, time: 0.410, data_time: 0.004, memory: 9389, loss_cls: 0.8139, loss_bbox: 0.3527, loss: 1.1666 2020-09-06 15:50:23,168 - INFO - Epoch [10][650/1389] lr: 0.00100, eta: 0:22:32, time: 0.371, data_time: 0.004, memory: 9389, loss_cls: 0.8266, loss_bbox: 0.3488, loss: 1.1753 2020-09-06 15:50:43,586 - INFO - Epoch [10][700/1389] lr: 0.00100, eta: 0:22:13, time: 0.408, data_time: 0.004, memory: 9389, loss_cls: 0.8016, loss_bbox: 0.3595, loss: 1.1611 2020-09-06 15:51:03,458 - INFO - Epoch [10][750/1389] lr: 0.00100, eta: 0:21:54, time: 0.397, data_time: 0.004, memory: 9389, loss_cls: 0.8425, loss_bbox: 0.3550, loss: 1.1976 2020-09-06 15:51:22,977 - INFO - Epoch [10][800/1389] lr: 0.00100, eta: 0:21:35, time: 0.390, data_time: 0.004, memory: 9389, loss_cls: 0.8159, loss_bbox: 0.3550, loss: 1.1709 2020-09-06 15:51:43,222 - INFO - Epoch [10][850/1389] lr: 0.00100, eta: 0:21:16, time: 0.405, data_time: 0.004, memory: 9389, loss_cls: 0.8193, loss_bbox: 0.3551, loss: 1.1744 2020-09-06 15:52:01,983 - INFO - Epoch [10][900/1389] lr: 0.00100, eta: 0:20:57, time: 0.375, data_time: 0.004, memory: 9389, loss_cls: 0.8093, loss_bbox: 0.3500, loss: 1.1592 2020-09-06 15:52:21,840 - INFO - Epoch [10][950/1389] lr: 0.00100, eta: 0:20:38, time: 0.397, data_time: 0.004, memory: 9389, loss_cls: 0.8140, loss_bbox: 0.3524, loss: 1.1664 2020-09-06 15:52:41,754 - INFO - Epoch [10][1000/1389] lr: 0.00100, eta: 0:20:19, time: 0.398, data_time: 0.004, memory: 9389, loss_cls: 0.8262, loss_bbox: 0.3572, loss: 1.1835 2020-09-06 15:53:01,719 - INFO - Epoch [10][1050/1389] lr: 0.00100, eta: 0:19:59, time: 0.399, data_time: 0.004, memory: 9389, loss_cls: 0.8303, loss_bbox: 0.3432, loss: 1.1735 2020-09-06 15:53:21,364 - INFO - Epoch [10][1100/1389] lr: 0.00100, eta: 0:19:40, time: 0.393, data_time: 0.004, memory: 9389, loss_cls: 0.8336, loss_bbox: 0.3508, loss: 1.1845 2020-09-06 15:53:40,835 - INFO - Epoch [10][1150/1389] lr: 0.00100, eta: 0:19:21, time: 0.389, data_time: 0.004, memory: 9389, loss_cls: 0.8290, loss_bbox: 0.3666, loss: 1.1956 2020-09-06 15:54:00,074 - INFO - Epoch [10][1200/1389] lr: 0.00100, eta: 0:19:02, time: 0.385, data_time: 0.004, memory: 9389, loss_cls: 0.8021, loss_bbox: 0.3513, loss: 1.1534 2020-09-06 15:54:19,252 - INFO - Epoch [10][1250/1389] lr: 0.00100, eta: 0:18:43, time: 0.384, data_time: 0.004, memory: 9389, loss_cls: 0.8440, loss_bbox: 0.3446, loss: 1.1885 2020-09-06 15:54:39,699 - INFO - Epoch [10][1300/1389] lr: 0.00100, eta: 0:18:24, time: 0.409, data_time: 0.004, memory: 9389, loss_cls: 0.8178, loss_bbox: 0.3488, loss: 1.1666 2020-09-06 15:55:01,630 - INFO - Epoch [10][1350/1389] lr: 0.00100, eta: 0:18:05, time: 0.439, data_time: 0.004, memory: 9389, loss_cls: 0.8315, loss_bbox: 0.3533, loss: 1.1847 2020-09-06 15:55:38,607 - INFO - Epoch [11][50/1389] lr: 0.00100, eta: 0:17:28, time: 0.408, data_time: 0.014, memory: 9389, loss_cls: 0.8377, loss_bbox: 0.3534, loss: 1.1911 2020-09-06 15:55:58,625 - INFO - Epoch [11][100/1389] lr: 0.00100, eta: 0:17:09, time: 0.400, data_time: 0.004, memory: 9389, loss_cls: 0.8358, loss_bbox: 0.3613, loss: 1.1971 2020-09-06 15:56:17,519 - INFO - Epoch [11][150/1389] lr: 0.00100, eta: 0:16:50, time: 0.378, data_time: 0.003, memory: 9389, loss_cls: 0.8168, loss_bbox: 0.3469, loss: 1.1637 2020-09-06 15:56:37,216 - INFO - Epoch [11][200/1389] lr: 0.00100, eta: 0:16:30, time: 0.394, data_time: 0.004, memory: 9389, loss_cls: 0.8114, loss_bbox: 0.3550, loss: 1.1664 2020-09-06 15:56:55,341 - INFO - Epoch [11][250/1389] lr: 0.00100, eta: 0:16:11, time: 0.362, data_time: 0.004, memory: 9389, loss_cls: 0.8627, loss_bbox: 0.3697, loss: 1.2323 2020-09-06 15:57:13,624 - INFO - Epoch [11][300/1389] lr: 0.00100, eta: 0:15:52, time: 0.366, data_time: 0.003, memory: 9389, loss_cls: 0.8362, loss_bbox: 0.3505, loss: 1.1867 2020-09-06 15:57:33,124 - INFO - Epoch [11][350/1389] lr: 0.00100, eta: 0:15:32, time: 0.390, data_time: 0.004, memory: 9389, loss_cls: 0.8329, loss_bbox: 0.3557, loss: 1.1887 2020-09-06 15:57:51,902 - INFO - Epoch [11][400/1389] lr: 0.00100, eta: 0:15:13, time: 0.376, data_time: 0.004, memory: 9389, loss_cls: 0.8067, loss_bbox: 0.3539, loss: 1.1607 2020-09-06 15:58:11,968 - INFO - Epoch [11][450/1389] lr: 0.00100, eta: 0:14:54, time: 0.401, data_time: 0.004, memory: 9389, loss_cls: 0.8465, loss_bbox: 0.3539, loss: 1.2005 2020-09-06 15:58:32,307 - INFO - Epoch [11][500/1389] lr: 0.00100, eta: 0:14:35, time: 0.407, data_time: 0.004, memory: 9389, loss_cls: 0.8487, loss_bbox: 0.3641, loss: 1.2128 2020-09-06 15:58:52,426 - INFO - Epoch [11][550/1389] lr: 0.00100, eta: 0:14:16, time: 0.402, data_time: 0.004, memory: 9389, loss_cls: 0.8420, loss_bbox: 0.3565, loss: 1.1985 2020-09-06 15:59:12,787 - INFO - Epoch [11][600/1389] lr: 0.00100, eta: 0:13:57, time: 0.407, data_time: 0.004, memory: 9389, loss_cls: 0.8177, loss_bbox: 0.3779, loss: 1.1957 2020-09-06 15:59:32,657 - INFO - Epoch [11][650/1389] lr: 0.00100, eta: 0:13:38, time: 0.397, data_time: 0.004, memory: 9389, loss_cls: 0.8139, loss_bbox: 0.3658, loss: 1.1797 2020-09-06 15:59:51,885 - INFO - Epoch [11][700/1389] lr: 0.00100, eta: 0:13:19, time: 0.385, data_time: 0.004, memory: 9389, loss_cls: 0.8160, loss_bbox: 0.3483, loss: 1.1643 2020-09-06 16:00:11,968 - INFO - Epoch [11][750/1389] lr: 0.00100, eta: 0:12:59, time: 0.402, data_time: 0.004, memory: 9389, loss_cls: 0.8057, loss_bbox: 0.3668, loss: 1.1725 2020-09-06 16:00:32,377 - INFO - Epoch [11][800/1389] lr: 0.00100, eta: 0:12:40, time: 0.408, data_time: 0.004, memory: 9389, loss_cls: 0.8388, loss_bbox: 0.3611, loss: 1.1999 2020-09-06 16:00:52,927 - INFO - Epoch [11][850/1389] lr: 0.00100, eta: 0:12:21, time: 0.411, data_time: 0.004, memory: 9389, loss_cls: 0.8317, loss_bbox: 0.3697, loss: 1.2014 2020-09-06 16:01:12,103 - INFO - Epoch [11][900/1389] lr: 0.00100, eta: 0:12:02, time: 0.384, data_time: 0.004, memory: 9389, loss_cls: 0.8212, loss_bbox: 0.3506, loss: 1.1718 2020-09-06 16:01:32,673 - INFO - Epoch [11][950/1389] lr: 0.00100, eta: 0:11:43, time: 0.411, data_time: 0.004, memory: 9389, loss_cls: 0.8283, loss_bbox: 0.3517, loss: 1.1800 2020-09-06 16:01:53,908 - INFO - Epoch [11][1000/1389] lr: 0.00100, eta: 0:11:24, time: 0.425, data_time: 0.004, memory: 9389, loss_cls: 0.8462, loss_bbox: 0.3503, loss: 1.1965 2020-09-06 16:02:14,607 - INFO - Epoch [11][1050/1389] lr: 0.00100, eta: 0:11:05, time: 0.414, data_time: 0.004, memory: 9389, loss_cls: 0.8222, loss_bbox: 0.3510, loss: 1.1731 2020-09-06 16:02:33,520 - INFO - Epoch [11][1100/1389] lr: 0.00100, eta: 0:10:46, time: 0.378, data_time: 0.004, memory: 9389, loss_cls: 0.8424, loss_bbox: 0.3642, loss: 1.2067 2020-09-06 16:02:52,364 - INFO - Epoch [11][1150/1389] lr: 0.00100, eta: 0:10:26, time: 0.377, data_time: 0.004, memory: 9389, loss_cls: 0.8431, loss_bbox: 0.3674, loss: 1.2105 2020-09-06 16:03:11,811 - INFO - Epoch [11][1200/1389] lr: 0.00100, eta: 0:10:07, time: 0.389, data_time: 0.004, memory: 9389, loss_cls: 0.8428, loss_bbox: 0.3731, loss: 1.2159 2020-09-06 16:03:32,495 - INFO - Epoch [11][1250/1389] lr: 0.00100, eta: 0:09:48, time: 0.414, data_time: 0.004, memory: 9389, loss_cls: 0.8302, loss_bbox: 0.3509, loss: 1.1810 2020-09-06 16:03:52,259 - INFO - Epoch [11][1300/1389] lr: 0.00100, eta: 0:09:29, time: 0.395, data_time: 0.004, memory: 9389, loss_cls: 0.8218, loss_bbox: 0.3622, loss: 1.1840 2020-09-06 16:04:12,975 - INFO - Epoch [11][1350/1389] lr: 0.00100, eta: 0:09:10, time: 0.414, data_time: 0.004, memory: 9389, loss_cls: 0.8370, loss_bbox: 0.3499, loss: 1.1869 2020-09-06 16:04:50,996 - INFO - Epoch [12][50/1389] lr: 0.00010, eta: 0:08:34, time: 0.425, data_time: 0.013, memory: 9389, loss_cls: 0.8230, loss_bbox: 0.3467, loss: 1.1696 2020-09-06 16:05:09,700 - INFO - Epoch [12][100/1389] lr: 0.00010, eta: 0:08:15, time: 0.374, data_time: 0.003, memory: 9389, loss_cls: 0.8261, loss_bbox: 0.3522, loss: 1.1783 2020-09-06 16:05:29,102 - INFO - Epoch [12][150/1389] lr: 0.00010, eta: 0:07:56, time: 0.388, data_time: 0.003, memory: 9389, loss_cls: 0.8242, loss_bbox: 0.3531, loss: 1.1773 2020-09-06 16:05:49,180 - INFO - Epoch [12][200/1389] lr: 0.00010, eta: 0:07:37, time: 0.402, data_time: 0.003, memory: 9389, loss_cls: 0.8515, loss_bbox: 0.3549, loss: 1.2064 2020-09-06 16:06:08,258 - INFO - Epoch [12][250/1389] lr: 0.00010, eta: 0:07:17, time: 0.382, data_time: 0.003, memory: 9389, loss_cls: 0.8377, loss_bbox: 0.3507, loss: 1.1884 2020-09-06 16:06:27,027 - INFO - Epoch [12][300/1389] lr: 0.00010, eta: 0:06:58, time: 0.375, data_time: 0.004, memory: 9389, loss_cls: 0.8165, loss_bbox: 0.3603, loss: 1.1769 2020-09-06 16:06:47,859 - INFO - Epoch [12][350/1389] lr: 0.00010, eta: 0:06:39, time: 0.417, data_time: 0.004, memory: 9389, loss_cls: 0.8219, loss_bbox: 0.3551, loss: 1.1770 2020-09-06 16:07:08,291 - INFO - Epoch [12][400/1389] lr: 0.00010, eta: 0:06:20, time: 0.409, data_time: 0.004, memory: 9389, loss_cls: 0.8258, loss_bbox: 0.3647, loss: 1.1905 2020-09-06 16:07:27,397 - INFO - Epoch [12][450/1389] lr: 0.00010, eta: 0:06:01, time: 0.382, data_time: 0.004, memory: 9389, loss_cls: 0.8078, loss_bbox: 0.3530, loss: 1.1608 2020-09-06 16:07:47,726 - INFO - Epoch [12][500/1389] lr: 0.00010, eta: 0:05:41, time: 0.407, data_time: 0.004, memory: 9389, loss_cls: 0.8184, loss_bbox: 0.3473, loss: 1.1657 2020-09-06 16:08:08,144 - INFO - Epoch [12][550/1389] lr: 0.00010, eta: 0:05:22, time: 0.408, data_time: 0.004, memory: 9389, loss_cls: 0.8686, loss_bbox: 0.3547, loss: 1.2233 2020-09-06 16:08:28,681 - INFO - Epoch [12][600/1389] lr: 0.00010, eta: 0:05:03, time: 0.411, data_time: 0.004, memory: 9389, loss_cls: 0.8152, loss_bbox: 0.3652, loss: 1.1804 2020-09-06 16:08:48,172 - INFO - Epoch [12][650/1389] lr: 0.00010, eta: 0:04:44, time: 0.390, data_time: 0.004, memory: 9389, loss_cls: 0.8276, loss_bbox: 0.3465, loss: 1.1742 2020-09-06 16:09:08,124 - INFO - Epoch [12][700/1389] lr: 0.00010, eta: 0:04:25, time: 0.399, data_time: 0.004, memory: 9389, loss_cls: 0.8105, loss_bbox: 0.3579, loss: 1.1684 2020-09-06 16:09:27,831 - INFO - Epoch [12][750/1389] lr: 0.00010, eta: 0:04:05, time: 0.394, data_time: 0.004, memory: 9389, loss_cls: 0.8294, loss_bbox: 0.3567, loss: 1.1861 2020-09-06 16:09:49,646 - INFO - Epoch [12][800/1389] lr: 0.00010, eta: 0:03:46, time: 0.436, data_time: 0.004, memory: 9389, loss_cls: 0.8462, loss_bbox: 0.3494, loss: 1.1956 2020-09-06 16:10:09,650 - INFO - Epoch [12][850/1389] lr: 0.00010, eta: 0:03:27, time: 0.400, data_time: 0.004, memory: 9389, loss_cls: 0.8061, loss_bbox: 0.3526, loss: 1.1587 2020-09-06 16:10:27,821 - INFO - Epoch [12][900/1389] lr: 0.00010, eta: 0:03:08, time: 0.363, data_time: 0.004, memory: 9389, loss_cls: 0.8038, loss_bbox: 0.3534, loss: 1.1572 2020-09-06 16:10:47,292 - INFO - Epoch [12][950/1389] lr: 0.00010, eta: 0:02:49, time: 0.389, data_time: 0.004, memory: 9389, loss_cls: 0.8403, loss_bbox: 0.3656, loss: 1.2059 2020-09-06 16:11:06,757 - INFO - Epoch [12][1000/1389] lr: 0.00010, eta: 0:02:29, time: 0.389, data_time: 0.004, memory: 9389, loss_cls: 0.8314, loss_bbox: 0.3694, loss: 1.2008 2020-09-06 16:11:26,767 - INFO - Epoch [12][1050/1389] lr: 0.00010, eta: 0:02:10, time: 0.400, data_time: 0.004, memory: 9389, loss_cls: 0.8136, loss_bbox: 0.3608, loss: 1.1744 2020-09-06 16:11:45,399 - INFO - Epoch [12][1100/1389] lr: 0.00010, eta: 0:01:51, time: 0.373, data_time: 0.004, memory: 9389, loss_cls: 0.8127, loss_bbox: 0.3558, loss: 1.1685 2020-09-06 16:12:05,506 - INFO - Epoch [12][1150/1389] lr: 0.00010, eta: 0:01:32, time: 0.402, data_time: 0.004, memory: 9389, loss_cls: 0.8531, loss_bbox: 0.3568, loss: 1.2099 2020-09-06 16:12:25,253 - INFO - Epoch [12][1200/1389] lr: 0.00010, eta: 0:01:12, time: 0.395, data_time: 0.004, memory: 9389, loss_cls: 0.8402, loss_bbox: 0.3745, loss: 1.2147 2020-09-06 16:12:44,406 - INFO - Epoch [12][1250/1389] lr: 0.00010, eta: 0:00:53, time: 0.383, data_time: 0.004, memory: 9389, loss_cls: 0.8262, loss_bbox: 0.3499, loss: 1.1761 2020-09-06 16:13:03,590 - INFO - Epoch [12][1300/1389] lr: 0.00010, eta: 0:00:34, time: 0.384, data_time: 0.004, memory: 9389, loss_cls: 0.8523, loss_bbox: 0.3554, loss: 1.2077 2020-09-06 16:13:23,588 - INFO - Epoch [12][1350/1389] lr: 0.00010, eta: 0:00:15, time: 0.400, data_time: 0.004, memory: 9389, loss_cls: 0.8153, loss_bbox: 0.3478, loss: 1.1631

hasanirtiza commented 4 years ago

Try 20-40 epochs.

msha096 commented 4 years ago

Try 20-40 epochs.

The LR is too large for one GPU. And that is the reason. After reducing LR, I got around 22% MR on reasonable when I train 12 epochs. How many epochs did you train to get around 14% MR ?

hasanirtiza commented 4 years ago

~20

123dddd commented 2 years ago

Try 20-40 epochs.

The LR is too large for one GPU. And that is the reason. After reducing LR, I got around 22% MR on reasonable when I train 12 epochs. How many epochs did you train to get around 14% MR ?

Hi, could you pls show the config here? I also have to choose a small LR for the training, the default LR is large for one GPU case.