HaoZhang534
commented
1 year ago I am using the DINO for training, When I check the log.txt, I found that it only kept a log of the training and no information about the evaluation.
Every time an epoch is trained, it is tested, but it is not clear why there is only information about the training in the log, but not about these tests? Like these:
IoU metric: bbox Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.136 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.355 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.065 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.125 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.214 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.263 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.357 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.443 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.386 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.439Only these:
{"train_lr": 9.999999999999128e-05, "train_class_error": 10.289773526515685, "train_loss": 10.081427183675487, "train_loss_ce_dn": 0.02354422877760819, "train_loss_bbox_dn": 0.14512152448590002, "train_loss_giou_dn": 0.4435547965939395, "train_loss_ce": 0.0900848628295791, "train_loss_bbox": 0.2033133311698915, "train_loss_giou": 0.5819766352843728, "train_loss_ce_0": 0.09858663630699945, "train_loss_bbox_0": 0.20667259970239438, "train_loss_giou_0": 0.5906689303772144, "train_loss_ce_dn_0": 0.03253694157097643, "train_loss_bbox_dn_0": 0.19445537988718492, "train_loss_giou_dn_0": 0.5651880545985231, "train_loss_ce_1": 0.09007208973138696, "train_loss_bbox_1": 0.20286299736785443, "train_loss_giou_1": 0.5821046600162035, "train_loss_ce_dn_1": 0.02017862054814205, "train_loss_bbox_dn_1": 0.14705833371947177, "train_loss_giou_dn_1": 0.4473134698282064, "train_loss_ce_2": 0.08932727297293415, "train_loss_bbox_2": 0.2028144848768655, "train_loss_giou_2": 0.5805991356224262, "train_loss_ce_dn_2": 0.019526467146330544, "train_loss_bbox_dn_2": 0.14466635804198383, "train_loss_giou_dn_2": 0.4414240188581193, "train_loss_ce_3": 0.08997658153358662, "train_loss_bbox_3": 0.20353458143816808, "train_loss_giou_3": 0.5817537180548027, "train_loss_ce_dn_3": 0.020651263656211387, "train_loss_bbox_dn_3": 0.1448027052562739, "train_loss_giou_dn_3": 0.44219723546332257, "train_loss_ce_4": 0.09002750413511551, "train_loss_bbox_4": 0.2033463555353698, "train_loss_giou_4": 0.5815244697538471, "train_loss_ce_dn_4": 0.02191437598789693, "train_loss_bbox_dn_4": 0.1449545008160634, "train_loss_giou_dn_4": 0.4428671849924295, "train_loss_ce_interm": 0.14654779242853863, "train_loss_bbox_interm": 0.21603473181889887, "train_loss_giou_interm": 0.6076423504597729, "train_loss_ce_dn_unscaled": 0.02354422877760819, "train_loss_bbox_dn_unscaled": 0.02902430487986099, "train_loss_giou_dn_unscaled": 0.22177739829696974, "train_loss_xy_dn_unscaled": 0.010142169331950692, "train_loss_hw_dn_unscaled": 0.018882135506381355, "train_cardinality_error_dn_unscaled": 196.70080180053452, "train_loss_ce_unscaled": 0.0900848628295791, "train_class_error_unscaled": 10.289773526515685, "train_loss_bbox_unscaled": 0.04066266622727079, "train_loss_giou_unscaled": 0.2909883176421864, "train_loss_xy_unscaled": 0.011668245515614887, "train_loss_hw_unscaled": 0.02899442072095734, "train_cardinality_error_unscaled": 899.0077366718244, "train_loss_ce_0_unscaled": 0.09858663630699945, "train_loss_bbox_0_unscaled": 0.041334519938592384, "train_loss_giou_0_unscaled": 0.2953344651886072, "train_loss_xy_0_unscaled": 0.011828351775696508, "train_loss_hw_0_unscaled": 0.0295061681549045, "train_cardinality_error_0_unscaled": 899.0077366718244, "train_loss_ce_dn_0_unscaled": 0.03253694157097643, "train_loss_bbox_dn_0_unscaled": 0.03889107594929687, "train_loss_giou_dn_0_unscaled": 0.28259402729926153, "train_loss_xy_dn_0_unscaled": 0.012192927245383425, "train_loss_hw_dn_0_unscaled": 0.02669814872454691, "train_cardinality_error_dn_0_unscaled": 196.6580391053594, "train_loss_ce_1_unscaled": 0.09007208973138696, "train_loss_bbox_1_unscaled": 0.04057259948520423, "train_loss_giou_1_unscaled": 0.29105233000810177, "train_loss_xy_1_unscaled": 0.011633653070762981, "train_loss_hw_1_unscaled": 0.028938946432716602, "train_cardinality_error_1_unscaled": 899.0077366718244, "train_loss_ce_dn_1_unscaled": 0.02017862054814205, "train_loss_bbox_dn_1_unscaled": 0.029411666722776165, "train_loss_giou_dn_1_unscaled": 0.2236567349141032, "train_loss_xy_dn_1_unscaled": 0.010172427561138554, "train_loss_hw_dn_1_unscaled": 0.019239239149683302, "train_cardinality_error_dn_1_unscaled": 196.70080180053452, "train_loss_ce_2_unscaled": 0.08932727297293415, "train_loss_bbox_2_unscaled": 0.04056289695902353, "train_loss_giou_2_unscaled": 0.2902995678112131, "train_loss_xy_2_unscaled": 0.011615480192556909, "train_loss_hw_2_unscaled": 0.02894741677183787, "train_cardinality_error_2_unscaled": 899.0077366718244, "train_loss_ce_dn_2_unscaled": 0.019526467146330544, "train_loss_bbox_dn_2_unscaled": 0.028933271577911635, "train_loss_giou_dn_2_unscaled": 0.22071200942905966, "train_loss_xy_dn_2_unscaled": 0.010068854263503922, "train_loss_hw_dn_2_unscaled": 0.01886441729051547, "train_cardinality_error_dn_2_unscaled": 196.70080180053452, "train_loss_ce_3_unscaled": 0.08997658153358662, "train_loss_bbox_3_unscaled": 0.040706916272960934, "train_loss_giou_3_unscaled": 0.29087685902740135, "train_loss_xy_3_unscaled": 0.011642910774473878, "train_loss_hw_3_unscaled": 0.029064005495997938, "train_cardinality_error_3_unscaled": 899.0077366718244, "train_loss_ce_dn_3_unscaled": 0.020651263656211387, "train_loss_bbox_dn_3_unscaled": 0.028960541043315805, "train_loss_giou_dn_3_unscaled": 0.22109861773166128, "train_loss_xy_dn_3_unscaled": 0.010095376100065992, "train_loss_hw_dn_3_unscaled": 0.01886516498782465, "train_cardinality_error_dn_3_unscaled": 196.70080180053452, "train_loss_ce_4_unscaled": 0.09002750413511551, "train_loss_bbox_4_unscaled": 0.0406692711186287, "train_loss_giou_4_unscaled": 0.29076223487692354, "train_loss_xy_4_unscaled": 0.011652468446915178, "train_loss_hw_4_unscaled": 0.029016802682030256, "train_cardinality_error_4_unscaled": 899.0077366718244, "train_loss_ce_dn_4_unscaled": 0.02191437598789693, "train_loss_bbox_dn_4_unscaled": 0.028990900129373796, "train_loss_giou_dn_4_unscaled": 0.22143359249621475, "train_loss_xy_dn_4_unscaled": 0.010118453057089426, "train_loss_hw_dn_4_unscaled": 0.018872447090739857, "train_cardinality_error_dn_4_unscaled": 196.70080180053452, "train_loss_ce_interm_unscaled": 0.14654779242853863, "train_loss_bbox_interm_unscaled": 0.043206946389037756, "train_loss_giou_interm_unscaled": 0.30382117522988644, "train_loss_xy_interm_unscaled": 0.012454071601941596, "train_loss_hw_interm_unscaled": 0.030752874787947703, "train_cardinality_error_interm_unscaled": 899.0077366718244, "test_class_error": 9.515765765765765, "test_loss": 6.273748900036554, "test_loss_bbox_dn": 0.0, "test_loss_giou_dn": 0.0, "test_loss_ce_dn": 0.0, "test_loss_ce": 0.09860444580315104, "test_loss_bbox": 0.1912704076584395, "test_loss_giou": 0.5973264330008008, "test_loss_ce_0": 0.102776809830814, "test_loss_bbox_0": 0.19208818674716796, "test_loss_giou_0": 0.5997700474753573, "test_loss_bbox_dn_0": 0.0, "test_loss_giou_dn_0": 0.0, "test_loss_ce_dn_0": 0.0, "test_loss_ce_1": 0.10399824881519512, "test_loss_bbox_1": 0.18827173477122644, "test_loss_giou_1": 0.5931415203119721, "test_loss_bbox_dn_1": 0.0, "test_loss_giou_dn_1": 0.0, "test_loss_ce_dn_1": 0.0, "test_loss_ce_2": 0.09758363557074239, "test_loss_bbox_2": 0.1901336608911856, "test_loss_giou_2": 0.5932897651517713, "test_loss_bbox_dn_2": 0.0, "test_loss_giou_dn_2": 0.0, "test_loss_ce_dn_2": 0.0, "test_loss_ce_3": 0.09537139786357124, "test_loss_bbox_3": 0.19121341114897314, "test_loss_giou_3": 0.5953284241930321, "test_loss_bbox_dn_3": 0.0, "test_loss_giou_dn_3": 0.0, "test_loss_ce_dn_3": 0.0, "test_loss_ce_4": 0.09409269005417062, "test_loss_bbox_4": 0.19049472951640686, "test_loss_giou_4": 0.5955235947695401, "test_loss_bbox_dn_4": 0.0, "test_loss_giou_dn_4": 0.0, "test_loss_ce_dn_4": 0.0, "test_loss_ce_interm": 0.15831829277371284, "test_loss_bbox_interm": 0.19643367112295385, "test_loss_giou_interm": 0.6087177919710541, "test_loss_bbox_dn_unscaled": 0.0, "test_loss_giou_dn_unscaled": 0.0, "test_loss_ce_dn_unscaled": 0.0, "test_loss_xy_dn_unscaled": 0.0, "test_loss_hw_dn_unscaled": 0.0, "test_cardinality_error_dn_unscaled": 0.0, "test_loss_ce_unscaled": 0.09860444580315104, "test_class_error_unscaled": 9.515765765765765, "test_loss_bbox_unscaled": 0.038254081614542045, "test_loss_giou_unscaled": 0.2986632165004004, "test_loss_xy_unscaled": 0.011340536810625513, "test_loss_hw_unscaled": 0.026913544835380134, "test_cardinality_error_unscaled": 898.9971846846847, "test_loss_ce_0_unscaled": 0.102776809830814, "test_loss_bbox_0_unscaled": 0.038417637341882326, "test_loss_giou_0_unscaled": 0.29988502373767867, "test_loss_xy_0_unscaled": 0.011315105334008197, "test_loss_hw_0_unscaled": 0.027102532017313212, "test_cardinality_error_0_unscaled": 898.9971846846847, "test_loss_bbox_dn_0_unscaled": 0.0, "test_loss_giou_dn_0_unscaled": 0.0, "test_loss_ce_dn_0_unscaled": 0.0, "test_loss_xy_dn_0_unscaled": 0.0, "test_loss_hw_dn_0_unscaled": 0.0, "test_cardinality_error_dn_0_unscaled": 0.0, "test_loss_ce_1_unscaled": 0.10399824881519512, "test_loss_bbox_1_unscaled": 0.037654346914391394, "test_loss_giou_1_unscaled": 0.29657076015598605, "test_loss_xy_1_unscaled": 0.011136157773770727, "test_loss_hw_1_unscaled": 0.026518189143767027, "test_cardinality_error_1_unscaled": 898.9971846846847, "test_loss_bbox_dn_1_unscaled": 0.0, "test_loss_giou_dn_1_unscaled": 0.0, "test_loss_ce_dn_1_unscaled": 0.0, "test_loss_xy_dn_1_unscaled": 0.0, "test_loss_hw_dn_1_unscaled": 0.0, "test_cardinality_error_dn_1_unscaled": 0.0, "test_loss_ce_2_unscaled": 0.09758363557074239, "test_loss_bbox_2_unscaled": 0.03802673215558333, "test_loss_giou_2_unscaled": 0.29664488257588567, "test_loss_xy_2_unscaled": 0.011161049845762685, "test_loss_hw_2_unscaled": 0.026865682306674285, "test_cardinality_error_2_unscaled": 898.9971846846847, "test_loss_bbox_dn_2_unscaled": 0.0, "test_loss_giou_dn_2_unscaled": 0.0, "test_loss_ce_dn_2_unscaled": 0.0, "test_loss_xy_dn_2_unscaled": 0.0, "test_loss_hw_dn_2_unscaled": 0.0, "test_cardinality_error_dn_2_unscaled": 0.0, "test_loss_ce_3_unscaled": 0.09537139786357124, "test_loss_bbox_3_unscaled": 0.038242682261887505, "test_loss_giou_3_unscaled": 0.29766421209651606, "test_loss_xy_3_unscaled": 0.011266453880690844, "test_loss_hw_3_unscaled": 0.026976228401123604, "test_cardinality_error_3_unscaled": 898.9971846846847, "test_loss_bbox_dn_3_unscaled": 0.0, "test_loss_giou_dn_3_unscaled": 0.0, "test_loss_ce_dn_3_unscaled": 0.0, "test_loss_xy_dn_3_unscaled": 0.0, "test_loss_hw_dn_3_unscaled": 0.0, "test_cardinality_error_dn_3_unscaled": 0.0, "test_loss_ce_4_unscaled": 0.09409269005417062, "test_loss_bbox_4_unscaled": 0.038098945984037944, "test_loss_giou_4_unscaled": 0.29776179738477004, "test_loss_xy_4_unscaled": 0.011269547806649998, "test_loss_hw_4_unscaled": 0.02682939816060736, "test_cardinality_error_4_unscaled": 898.9971846846847, "test_loss_bbox_dn_4_unscaled": 0.0, "test_loss_giou_dn_4_unscaled": 0.0, "test_loss_ce_dn_4_unscaled": 0.0, "test_loss_xy_dn_4_unscaled": 0.0, "test_loss_hw_dn_4_unscaled": 0.0, "test_cardinality_error_dn_4_unscaled": 0.0, "test_loss_ce_interm_unscaled": 0.15831829277371284, "test_loss_bbox_interm_unscaled": 0.039286734202776, "test_loss_giou_interm_unscaled": 0.30435889598552707, "test_loss_xy_interm_unscaled": 0.01193497632105776, "test_loss_hw_interm_unscaled": 0.027351757872279163, "test_cardinality_error_interm_unscaled": 898.9971846846847, "test_coco_eval_bbox": [0.3414813500256674, 0.820382486408012, 0.18570287418414488, -1.0, 0.2804167932036041, 0.36790075053906207, 0.45730272339397204, 0.49702271610610027, 0.5393201735524203, -1.0, 0.4831834706630447, 0.5571628397801337], "best_res": 0.3513627732678881, "best_ep": 2, "epoch": 3, "n_parameters": 46627092, "now_time": "2023-01-13 08:32:21.177336", "epoch_time": "0:46:44"}
This information is printed in stdout. You can redirect stdout to a file.
I am using the DINO for training, When I check the log.txt, I found that it only kept a log of the training and no information about the evaluation.
Every time an epoch is trained, it is tested, but it is not clear why there is only information about the training in the log, but not about these tests? Like these:
Only these: