How to Train to Achieve the Result in the Paper

[zranwang]

I trained the model on the dataset COCO-Standard 10%, but I couldn' t reach the result of AP=35.11 given in the paper, and my best result is as follows, What's wrong with me?（I used the provided configuration file without any modifications.）

AP	AP50	AP75	APs	APm	APl
34.519	53.205	37.222	19.698	37.772	43.981

[ZRandomize]

hi, please test the Teacher (the EMA model)

[zranwang]

Yes, I test the Teacher (the EMA model).

[ZRandomize]

can you offer the full log file?

[zranwang]

oh, I'm sorry that I can't upload the log file due to personal network environment problems, but I can provide key log information.

[ZRandomize]

please offer the full config log and the history evaluation results

[zranwang]

2023-02-23 14:04:54.776	INFO	cvpods.engine.setup:default_setup:137 - Rank of current process: 0. World size: 8 2023-02-23 14:04:56.191	INFO	cvpods.engine.setup:default_setup:139 - Environment info:

sys.platform linux Python 3.8.5 (default, Sep 4 2020, 07:30:14) [GCC 7.3.0] numpy 1.20.3 cvpods 0.1 @/opt/schedule-train/algorithm/DenseTeacher/cvpods/cvpods cvpods compiler GCC 7.5 cvpods CUDA compiler 10.1 cvpods arch flags sm_70 cvpods_ENV_MODULE PyTorch 1.8.1+cu102 @/root/miniconda3/lib/python3.8/site-packages/torch PyTorch debug build False CUDA available True GPU 0,1,2,3,4,5,6,7 Tesla V100-SXM2-32GB CUDA_HOME /usr/local/cuda NVCC Cuda compilation tools, release 10.1, V10.1.243 Pillow 6.2.1 torchvision 0.9.1+cu102 @/root/miniconda3/lib/python3.8/site-packages/torchvision torchvision arch flags sm_35, sm_50, sm_60, sm_70, sm_75 cv2 4.5.4-dev

---

PyTorch built with:

• GCC 7.3
• C++ Version: 201402
• Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
• Intel(R) MKL-DNN v1.7.0 (Git Hash 7aed236906b1f7a05c0917e5257a1af05e9ff683)
• OpenMP 201511 (a.k.a. OpenMP 4.5)
• NNPACK is enabled
• CPU capability usage: AVX2
• CUDA Runtime 10.2
• NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70 2023-02-23 14:07:00.676 | INFO | __mp_main__:main:103 - Running with full config: ╒═══════════════════════╤═════════════════════════════════════════════════════════════════════════════════════╕ │ config params │ values │ ╞═══════════════════════╪═════════════════════════════════════════════════════════════════════════════════════╡ │ MODEL │ {'ANCHOR_GENERATOR': {'ANGLES': [[-90, 0, 90]], │ │ │ 'ASPECT_RATIOS': [[0.5, 1.0, 2.0]], │ │ │ 'OFFSET': 0.0, │ │ │ 'SIZES': [[32, 64, 128, 256, 512]]}, │ │ │ 'AS_PRETRAIN': False, │ │ │ 'BACKBONE': {'FREEZE_AT': 2}, │ │ │ 'DDP_BACKEND': 'torch', │ │ │ 'DEVICE': 'cuda', │ │ │ 'FCOS': {'BBOX_REG_WEIGHTS': [1.0, 1.0, 1.0, 1.0], │ │ │ 'CENTERNESS_ON_REG': True, │ │ │ 'CENTER_SAMPLING_RADIUS': 1.5, │ │ │ 'FOCAL_LOSS_ALPHA': 0.25, │ │ │ 'FOCAL_LOSS_GAMMA': 2.0, │ │ │ 'FPN_STRIDES': [8, 16, 32, 64, 128], │ │ │ 'IN_FEATURES': ['p3', 'p4', 'p5', 'p6', 'p7'], │ │ │ 'IOU_LOSS_TYPE': 'giou', │ │ │ 'NMS_THRESH_TEST': 0.6, │ │ │ 'NORM_REG_TARGETS': True, │ │ │ 'NORM_SYNC': True, │ │ │ 'NUM_CLASSES': 80, │ │ │ 'NUM_CONVS': 4, │ │ │ 'OBJECT_SIZES_OF_INTEREST': [[-1, 64], │ │ │ [64, 128], │ │ │ [128, 256], │ │ │ [256, 512], │ │ │ [512, inf]], │ │ │ 'PRIOR_PROB': 0.01, │ │ │ 'QUALITY_BRANCH': 'iou', │ │ │ 'SCORE_THRESH_TEST': 0.05, │ │ │ 'TOPK_CANDIDATES_TEST': 1000}, │ │ │ 'FPN': {'BLOCK_IN_FEATURES': 'p5', │ │ │ 'FUSE_TYPE': 'sum', │ │ │ 'IN_FEATURES': ['res3', 'res4', 'res5'], │ │ │ 'NORM': '', │ │ │ 'OUT_CHANNELS': 256}, │ │ │ 'KEYPOINT_ON': False, │ │ │ 'LOAD_PROPOSALS': False, │ │ │ 'MASK_ON': False, │ │ │ 'NMS_TYPE': 'normal', │ │ │ 'PIXEL_MEAN': [103.53, 116.28, 123.675], │ │ │ 'PIXEL_STD': [1.0, 1.0, 1.0], │ │ │ 'RESNETS': {'ACTIVATION': {'INPLACE': True, 'NAME': 'ReLU'}, │ │ │ 'DEEP_STEM': False, │ │ │ 'DEPTH': 50, │ │ │ 'NORM': 'FrozenBN', │ │ │ 'NUM_CLASSES': None, │ │ │ 'NUM_GROUPS': 1, │ │ │ 'OUT_FEATURES': ['res3', 'res4', 'res5'], │ │ │ 'RES2_OUT_CHANNELS': 256, │ │ │ 'RES5_DILATION': 1, │ │ │ 'STEM_OUT_CHANNELS': 64, │ │ │ 'STRIDE_IN_1X1': True, │ │ │ 'WIDTH_PER_GROUP': 64, │ │ │ 'ZERO_INIT_RESIDUAL': False}, │ │ │ 'SHIFT_GENERATOR': {'NUM_SHIFTS': 1, 'OFFSET': 0.0}, │ │ │ 'WEIGHTS': '../checkpoints/R-50.pkl'} │ ├───────────────────────┼─────────────────────────────────────────────────────────────────────────────────────┤ │ INPUT │ {'AUG': {'TEST_PIPELINES': [('ResizeShortestEdge', │ │ │ {'max_size': 1333, │ │ │ 'sample_style': 'choice', │ │ │ 'short_edge_length': 800})], │ │ │ 'TRAIN_PIPELINES': {'SUPERVISED': [<class 'augmentations.WeakAug'>, │ │ │ {'max_size': 1333, │ │ │ 'sample_style': 'choice', │ │ │ 'short_edge_length': [640, │ │ │ 672, │ │ │ 704, │ │ │ 736, │ │ │ 768, │ │ │ 800]}], │ │ │ 'UNSUPERVISED': [<class 'augmentations.StrongAug'>]}}, │ │ │ 'FORMAT': 'BGR', │ │ │ 'MASK_FORMAT': 'polygon'} │ ├───────────────────────┼─────────────────────────────────────────────────────────────────────────────────────┤ │ DATASETS │ {'CUSTOM_TYPE': ['ConcatDataset', {}], │ │ │ 'PRECOMPUTED_PROPOSAL_TOPK_TEST': 1000, │ │ │ 'PRECOMPUTED_PROPOSAL_TOPK_TRAIN': 2000, │ │ │ 'PROPOSAL_FILES_TEST': [], │ │ │ 'PROPOSAL_FILES_TRAIN': [], │ │ │ 'SUPERVISED': [(<class 'dataset.PartialCOCO'>, │ │ │ {'percentage': 10, │ │ │ 'seed': 1, │ │ │ 'sup_file': '../COCO_Division/COCO_supervision.txt', │ │ │ 'supervised': True})], │ │ │ 'TEST': ['coco_2017_val'], │ │ │ 'TRAIN': [], │ │ │ 'UNSUPERVISED': [(<class 'dataset.PartialCOCO'>, │ │ │ {'percentage': 10, │ │ │ 'seed': 1, │ │ │ 'sup_file': '../COCO_Division/COCO_supervision.txt', │ │ │ 'supervised': False})]} │ ├───────────────────────┼─────────────────────────────────────────────────────────────────────────────────────┤ │ DATALOADER │ {'ASPECT_RATIO_GROUPING': True, │ │ │ 'ENABLE_INF_SAMPLER': True, │ │ │ 'FILTER_EMPTY_ANNOTATIONS': True, │ │ │ 'NUM_WORKERS': 4, │ │ │ 'REPEAT_THRESHOLD': 0.0, │ │ │ 'SAMPLER_TRAIN': 'DistributedGroupSampler'} │ ├───────────────────────┼─────────────────────────────────────────────────────────────────────────────────────┤ │ TRAINER │ {'DISTILL': {'GAMMA': 2.0, │ │ │ 'RATIO': 0.01, │ │ │ 'SUPPRESS': 'linear', │ │ │ 'SUP_WEIGHT': 1, │ │ │ 'UNSUP_WEIGHT': 1, │ │ │ 'WEIGHTS': {'DELTAS': 1.0, 'LOGITS': 4.0, 'QUALITY': 1.0}}, │ │ │ 'EMA': {'DECAY_FACTOR': 0.9996, │ │ │ 'FAKE': False, │ │ │ 'START_STEPS': 3000, │ │ │ 'UPDATE_STEPS': 1}, │ │ │ 'FP16': {'ENABLED': False, 'OPTS': {'OPT_LEVEL': 'O1'}, 'TYPE': 'APEX'}, │ │ │ 'NAME': 'SemiRunner', │ │ │ 'SSL': {'BURN_IN_STEPS': 5000}, │ │ │ 'WINDOW_SIZE': 20} │ ├───────────────────────┼─────────────────────────────────────────────────────────────────────────────────────┤ │ SOLVER │ {'BATCH_SUBDIVISIONS': 1, │ │ │ 'CHECKPOINT_PERIOD': 5000, │ │ │ 'CLIP_GRADIENTS': {'CLIP_TYPE': 'value', │ │ │ 'CLIP_VALUE': 1.0, │ │ │ 'ENABLED': True, │ │ │ 'NORM_TYPE': 2.0}, │ │ │ 'IMS_PER_BATCH': 16, │ │ │ 'IMS_PER_DEVICE': 2, │ │ │ 'LR_SCHEDULER': {'EPOCH_ITERS': -1, │ │ │ 'GAMMA': 0.1, │ │ │ 'MAX_EPOCH': None, │ │ │ 'MAX_ITER': 180000, │ │ │ 'NAME': 'WarmupMultiStepLR', │ │ │ 'STEPS': [179995], │ │ │ 'WARMUP_FACTOR': 0.001, │ │ │ 'WARMUP_ITERS': 1000, │ │ │ 'WARMUP_METHOD': 'linear'}, │ │ │ 'OPTIMIZER': {'BASE_LR': 0.01, │ │ │ 'BIAS_LR_FACTOR': 1.0, │ │ │ 'MOMENTUM': 0.9, │ │ │ 'NAME': 'D2SGD', │ │ │ 'WEIGHT_DECAY': 0.0001, │ │ │ 'WEIGHT_DECAY_NORM': 0.0}} │ ├───────────────────────┼─────────────────────────────────────────────────────────────────────────────────────┤ │ TEST │ {'AUG': {'ENABLED': False, │ │ │ 'EXTRA_SIZES': [], │ │ │ 'FLIP': True, │ │ │ 'MAX_SIZE': 4000, │ │ │ 'MIN_SIZES': [400, 500, 600, 700, 800, 900, 1000, 1100, 1200], │ │ │ 'SCALE_FILTER': False, │ │ │ 'SCALE_RANGES': []}, │ │ │ 'DETECTIONS_PER_IMAGE': 100, │ │ │ 'EVAL_PERIOD': 2000, │ │ │ 'EXPECTED_RESULTS': [], │ │ │ 'KEYPOINT_OKS_SIGMAS': [], │ │ │ 'ON_FILES': False, │ │ │ 'PRECISE_BN': {'ENABLED': False, 'NUM_ITER': 200}} │ ├───────────────────────┼─────────────────────────────────────────────────────────────────────────────────────┤ │ OUTPUT_DIR │ 'outputs' │ ├───────────────────────┼─────────────────────────────────────────────────────────────────────────────────────┤ │ SEED │ 56207473 │ ├───────────────────────┼─────────────────────────────────────────────────────────────────────────────────────┤ │ CUDNN_BENCHMARK │ False │ ├───────────────────────┼─────────────────────────────────────────────────────────────────────────────────────┤ │ VIS_PERIOD │ 0 │ ├───────────────────────┼─────────────────────────────────────────────────────────────────────────────────────┤ │ GLOBAL │ {'CLEARML': {'ENABLE': False, │ │ │ 'OUTPUT_URI': None, │ │ │ 'PROJECT_NAME': 'cvpods', │ │ │ 'TAGS': None, │ │ │ 'TASK_NAME': None}, │ │ │ 'DUMP_TEST': False, │ │ │ 'DUMP_TRAIN': True, │ │ │ 'HACK': 1.0, │ │ │ 'LOG_INTERVAL': 10} │ ├───────────────────────┼─────────────────────────────────────────────────────────────────────────────────────┤ │ build_backbone │ <function build_backbone at 0x7fabbfebbc10> │ ├───────────────────────┼─────────────────────────────────────────────────────────────────────────────────────┤ │ build_shift_generator │ <function build_shift_generator at 0x7fabbfe61670> │ ╘═══════════════════════╧═════════════════════════════════════════════════════════════════════════════════════╛ 2023-02-23 14:07:00.681 | INFO | __mp_main__:main:105 - different config with base class: ╒═══════════════════════╤═════════════════════════════════════════════════════════════════════════════════════╕ │ config params │ values │ ╞═══════════════════════╪═════════════════════════════════════════════════════════════════════════════════════╡ │ MODEL │ {'FCOS': {'CENTERNESS_ON_REG': True, │ │ │ 'CENTER_SAMPLING_RADIUS': 1.5, │ │ │ 'IOU_LOSS_TYPE': 'giou', │ │ │ 'NORM_REG_TARGETS': True, │ │ │ 'QUALITY_BRANCH': 'iou'}, │ │ │ 'RESNETS': {'DEPTH': 50}, │ │ │ 'WEIGHTS': '../checkpoints/R-50.pkl'} │ ├───────────────────────┼─────────────────────────────────────────────────────────────────────────────────────┤ │ INPUT │ {'AUG': {'TRAIN_PIPELINES': {'SUPERVISED': [<class 'augmentations.WeakAug'>, │ │ │ {'max_size': 1333, │ │ │ 'sample_style': 'choice', │ │ │ 'short_edge_length': [640, │ │ │ 672, │ │ │ 704, │ │ │ 736, │ │ │ 768, │ │ │ 800]}], │ │ │ 'UNSUPERVISED': [<class 'augmentations.StrongAug'>]}}} │ ├───────────────────────┼─────────────────────────────────────────────────────────────────────────────────────┤ │ DATASETS │ {'SUPERVISED': [(<class 'dataset.PartialCOCO'>, │ │ │ {'percentage': 10, │ │ │ 'seed': 1, │ │ │ 'sup_file': '../COCO_Division/COCO_supervision.txt', │ │ │ 'supervised': True})], │ │ │ 'TEST': ['coco_2017_val'], │ │ │ 'UNSUPERVISED': [(<class 'dataset.PartialCOCO'>, │ │ │ {'percentage': 10, │ │ │ 'seed': 1, │ │ │ 'sup_file': '../COCO_Division/COCO_supervision.txt', │ │ │ 'supervised': False})]} │ ├───────────────────────┼─────────────────────────────────────────────────────────────────────────────────────┤ │ DATALOADER │ {'NUM_WORKERS': 4} │ ├───────────────────────┼─────────────────────────────────────────────────────────────────────────────────────┤ │ TRAINER │ {'DISTILL': {'GAMMA': 2.0, │ │ │ 'RATIO': 0.01, │ │ │ 'SUPPRESS': 'linear', │ │ │ 'SUP_WEIGHT': 1, │ │ │ 'UNSUP_WEIGHT': 1, │ │ │ 'WEIGHTS': {'DELTAS': 1.0, 'LOGITS': 4.0, 'QUALITY': 1.0}}, │ │ │ 'EMA': {'DECAY_FACTOR': 0.9996, │ │ │ 'FAKE': False, │ │ │ 'START_STEPS': 3000, │ │ │ 'UPDATE_STEPS': 1}, │ │ │ 'NAME': 'SemiRunner', │ │ │ 'SSL': {'BURN_IN_STEPS': 5000}} │ ├───────────────────────┼─────────────────────────────────────────────────────────────────────────────────────┤ │ SOLVER │ {'CLIP_GRADIENTS': {'ENABLED': True}, │ │ │ 'LR_SCHEDULER': {'EPOCH_ITERS': -1, 'MAX_ITER': 180000, 'STEPS': [179995]}, │ │ │ 'OPTIMIZER': {'BASE_LR': 0.01}} │ ├───────────────────────┼─────────────────────────────────────────────────────────────────────────────────────┤ │ TEST │ {'EVAL_PERIOD': 2000} │ ├───────────────────────┼─────────────────────────────────────────────────────────────────────────────────────┤ │ OUTPUT_DIR │ 'outputs' │ ├───────────────────────┼─────────────────────────────────────────────────────────────────────────────────────┤ │ SEED │ 56207473 │ ├───────────────────────┼─────────────────────────────────────────────────────────────────────────────────────┤ │ GLOBAL │ {'LOG_INTERVAL': 10} │ ├───────────────────────┼─────────────────────────────────────────────────────────────────────────────────────┤ │ build_backbone │ <function build_backbone at 0x7fabbfebbc10> │ ├───────────────────────┼─────────────────────────────────────────────────────────────────────────────────────┤ │ build_shift_generator │ <function build_shift_generator at 0x7fabbfe61670> │ ╘═══════════════════════╧═════════════════════════════════════════════════════════════════════════════════════╛

2023-02-23 14:19:33.378	INFO	runner:test_and_save_ema_results:142 - Evaluating: EMA 2023-02-23 14:20:37.690	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
0.000	0.000	0.000	0.000	0.000	0.000

2023-02-23 14:34:23.947	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
17.519	31.243	17.486	8.981	20.410	23.056

2023-02-23 14:53:15.495	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
20.719	35.841	21.095	10.592	23.742	26.901

2023-02-23 15:17:09.620	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
22.681	38.523	23.571	11.876	25.539	29.117

2023-02-23 15:41:01.502	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
23.959	40.206	24.961	12.610	27.005	30.500

2023-02-23 16:04:28.905	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
24.967	41.445	26.272	13.852	27.989	31.627

2023-02-23 16:27:54.270	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
25.880	42.690	27.194	13.831	29.113	32.653

2023-02-23 16:51:29.613	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
26.663	43.757	28.180	14.608	29.767	33.601

2023-02-23 17:15:03.982	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
27.422	44.594	29.118	14.842	30.525	34.651

2023-02-23 17:38:58.236	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
28.142	45.514	29.917	15.210	31.289	35.643

2023-02-23 18:02:59.018	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
28.692	46.328	30.475	15.547	31.845	36.326

2023-02-23 18:27:27.747	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
29.225	46.963	31.042	15.726	32.296	37.019

2023-02-23 18:51:41.811	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
29.664	47.515	31.669	16.029	32.638	37.726

2023-02-23 19:15:57.196	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
30.092	48.097	32.150	16.291	32.987	38.301

2023-02-23 19:39:54.444	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
30.441	48.555	32.316	16.635	33.361	38.694

2023-02-23 20:04:04.683	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
30.756	49.027	32.760	17.047	33.672	39.122

2023-02-23 20:27:52.081	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
31.104	49.398	33.281	17.557	34.050	39.554

2023-02-23 20:51:49.600	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
31.396	49.785	33.518	17.729	34.464	39.925

2023-02-23 21:15:56.587	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
31.678	49.921	33.937	17.919	34.726	40.141

2023-02-23 21:39:52.919	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
31.897	50.245	34.039	18.103	34.806	40.311

2023-02-23 22:04:28.469	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
32.076	50.414	34.277	18.208	35.037	41.123

2023-02-23 22:28:55.524	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
32.269	50.685	34.489	18.222	35.330	41.314

2023-02-23 22:53:39.985	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
32.391	50.935	34.510	17.865	35.604	41.475

2023-02-23 23:18:08.452	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
32.563	51.018	34.698	17.965	35.806	41.568

2023-02-23 23:42:54.143	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
32.659	51.209	34.868	18.033	35.870	41.774

2023-02-24 00:07:03.809	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
32.807	51.369	35.025	18.068	36.146	41.949

2023-02-24 00:31:32.331	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
32.967	51.481	35.284	18.508	36.345	42.181

2023-02-24 00:56:06.253	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
33.058	51.570	35.385	18.587	36.356	42.325

2023-02-24 01:20:59.358	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
33.115	51.664	35.527	18.628	36.338	42.524

2023-02-24 01:46:03.849	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
33.182	51.703	35.440	18.541	36.519	42.593

2023-02-24 02:11:14.076	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
33.230	51.697	35.541	18.385	36.535	42.874

2023-02-24 02:36:30.175	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
33.384	51.872	35.888	18.755	36.450	43.061

2023-02-24 03:01:32.966	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
33.427	51.842	36.080	18.691	36.501	43.103

2023-02-24 03:26:20.745	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
33.555	52.031	36.231	18.604	36.718	42.944

2023-02-24 03:51:07.817	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
33.626	52.172	36.110	18.538	36.901	42.437

2023-02-24 04:16:18.605	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
33.677	52.143	36.208	18.589	36.805	42.723

2023-02-24 04:41:23.969	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
33.684	52.191	36.119	18.745	37.007	42.800

2023-02-24 05:06:58.397	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
33.837	52.379	36.174	18.503	37.169	43.079

2023-02-24 05:32:25.041	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
33.883	52.296	36.214	18.702	37.156	43.052

2023-02-24 05:58:01.350	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
33.852	52.366	36.213	18.545	37.128	42.995

2023-02-24 06:23:22.724	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
33.835	52.352	36.284	18.428	37.115	42.898

2023-02-24 06:48:38.466	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
33.874	52.451	36.313	18.609	37.221	43.084

2023-02-24 07:13:59.622	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
33.944	52.475	36.498	18.900	37.218	42.887

2023-02-24 07:39:22.376	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.031	52.583	36.702	18.876	37.380	42.979

2023-02-24 08:04:55.782	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.127	52.679	36.715	18.709	37.538	43.158

2023-02-24 08:30:21.470	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.133	52.758	36.760	18.801	37.642	43.290

2023-02-24 08:56:13.877	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.097	52.724	36.629	18.696	37.543	43.393

2023-02-24 09:21:58.501	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.169	52.766	36.659	18.651	37.633	43.511

2023-02-24 09:47:52.513	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.199	52.782	36.763	18.561	37.582	43.459

2023-02-24 10:13:13.087	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.238	52.715	36.905	18.676	37.549	43.583

2023-02-24 10:38:41.427	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.311	52.721	36.971	18.821	37.578	43.628

2023-02-24 11:04:22.560	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.340	52.751	36.990	18.749	37.593	43.587

2023-02-24 11:30:02.377	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.361	52.772	37.189	18.889	37.783	43.441

2023-02-24 11:55:43.074	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.313	52.709	37.199	18.808	37.685	43.663

2023-02-24 12:21:53.472	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.359	52.772	37.162	18.547	37.665	43.692

2023-02-24 12:47:47.723	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.420	52.843	37.185	18.589	37.759	43.791

2023-02-24 13:14:05.629	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.371	52.977	37.007	18.629	37.608	43.955

2023-02-24 13:39:56.353	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.401	53.004	37.024	18.780	37.665	44.060

2023-02-24 14:05:48.075	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.413	53.091	36.836	18.801	37.482	43.936

2023-02-24 14:31:55.191	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.435	53.056	36.790	18.738	37.705	43.746

2023-02-24 14:57:56.404	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.509	53.098	36.902	19.063	37.668	43.719

2023-02-24 15:24:13.663	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.432	53.071	36.720	19.079	37.649	43.657

2023-02-24 15:50:41.306	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.346	52.928	36.778	19.076	37.623	43.722

2023-02-24 16:16:56.762	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.459	52.994	36.945	19.261	37.887	43.645

2023-02-24 16:43:07.267	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.447	53.021	37.085	19.535	37.857	43.336

2023-02-24 17:09:30.098	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.449	52.980	36.989	19.467	37.750	43.562

2023-02-24 17:35:31.747	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.440	53.055	37.087	19.629	37.774	43.662

2023-02-24 18:02:00.564	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.519	53.205	37.222	19.698	37.772	43.981

2023-02-24 18:28:40.685	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.545	53.247	37.217	19.433	37.776	43.942

2023-02-24 18:55:17.708	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.448	53.053	36.976	19.301	37.662	44.103

2023-02-24 19:22:26.150	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.502	53.094	36.950	19.186	37.550	43.889

2023-02-24 19:48:52.466	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.563	53.160	37.071	19.202	37.659	43.877

2023-02-24 20:15:32.536	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.474	53.151	36.814	19.481	37.600	44.219

2023-02-24 20:41:49.933	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.464	53.021	36.848	19.157	37.554	44.351

2023-02-24 21:08:26.357	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.484	53.022	36.852	19.111	37.687	44.071

2023-02-24 21:35:10.100	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.459	53.060	36.835	19.105	37.768	43.927

2023-02-24 22:01:58.302	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.549	53.128	36.971	19.246	37.902	44.075

2023-02-24 22:29:16.758	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.535	53.033	36.944	19.750	37.899	43.908

2023-02-24 22:56:17.983	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.626	53.253	36.829	19.652	37.905	44.143

2023-02-24 23:23:30.047	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.589	53.208	36.844	19.299	37.993	44.278

2023-02-24 23:50:25.424	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.639	53.163	37.059	19.445	37.785	44.304

2023-02-25 00:17:14.893	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.647	53.267	36.925	19.285	37.847	44.336

2023-02-25 00:44:30.016	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.657	53.215	36.989	19.186	37.945	44.671

2023-02-25 01:11:41.911	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.634	53.192	36.892	19.090	37.858	44.612

2023-02-25 01:39:18.500	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.707	53.252	36.978	19.116	37.876	44.509

2023-02-25 02:07:05.581	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.736	53.175	37.031	19.048	37.988	44.237

2023-02-25 02:34:58.286	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.717	53.146	37.095	19.043	37.993	44.245

2023-02-25 03:02:41.780	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.773	53.188	37.138	19.352	37.971	44.231

2023-02-25 03:30:14.178	INFO	cvpods.evaluation.coco_evaluation:_derive_coco_results:318 - Evaluation results for bbox:	AP	AP50	AP75	APs	APm	APl
34.759	53.284	37.155	19.369	38.029	44.326

[ZRandomize]

Seems like nothing goes wrong. This might be caused by different GPU (2080ti & V100) or random states.