MaskDino bbox

[rbumbi]

Hello,

first, I would to thank you for your excellent work; your network is impressive. Could I ask you for your advice about the proper train parameters for detection? When I train on a custom dataset with two classes, the evaluation results of segmentation are fine (about AP 80 after the second epoch). Still, detection results, bbox, are worse (about AP 58 after the second epoch).

Thank you very much

Best Regards

Roman Bumbalek

Here is config:

dataloader: evaluator: {target: detectron2.evaluation.COCOEvaluator, dataset_name: '${..test.dataset.names}', output_dir: ./output/dab_detr_r50_50ep} test: target: detectron2.data.build_detection_test_loader dataset: {target: detectron2.data.get_detection_dataset_dicts, filter_empty: false, names: cow_val} mapper: target: detrex.data.COCOInstanceNewBaselineDatasetMapper augmentation:

• {target: detectron2.data.transforms.ResizeShortestEdge, max_size: 1333, short_edge_length: 800} image_format: RGB is_train: false num_workers: 4 train: target: detectron2.data.build_detection_train_loader dataset: {target: detectron2.data.get_detection_dataset_dicts, names: cow_train} mapper: target: detrex.data.COCOInstanceNewBaselineDatasetMapper augmentation: {target: detrex.data.dataset_mappers.coco_instance_new_baseline_dataset_mapper.build_transform_gen, image_size: 720, max_scale: 2.0, min_scale: 0.1, random_flip: horizontal} image_format: RGB is_train: true num_workers: 4 total_batch_size: 4 lr_multiplier: target: detectron2.solver.WarmupParamScheduler scheduler: target: fvcore.common.param_scheduler.MultiStepParamScheduler milestones: [327778, 355092] values: [1.0, 0.1] warmup_factor: 1.0 warmup_length: 2.711864406779661e-05 model: target: projects.maskdino.maskdino.MaskDINO backbone: target: detrex.modeling.ResNet freeze_at: 1 out_features: [res2, res3, res4, res5] stages: {target: detrex.modeling.ResNet.make_default_stages, depth: 50, norm: FrozenBN, stride_in_1x1: false} stem: {target: detrex.modeling.BasicStem, in_channels: 3, norm: FrozenBN, out_channels: 64} criterion: target: projects.maskdino.modeling.criterion.SetCriterion dn: seg dn_losses: [labels, masks, boxes] eos_coef: 0.1 importance_sample_ratio: 0.75 losses: [labels, masks, boxes] matcher: {target: projects.maskdino.modeling.matcher.HungarianMatcher, cost_box: 5.0, cost_class: 4.0, cost_dice: 5.0, cost_giou: 2.0, cost_mask: 5.0, num_points: 12544, panoptic_on: '${..panoptic_on}'} num_classes: ${..sem_seg_head.num_classes} num_points: 12544 oversample_ratio: 3.0 panoptic_on: ${..panoptic_on} semantic_ce_loss: false weight_dict: {loss_bbox: 5.0, loss_bbox_0: 5.0, loss_bbox_1: 5.0, loss_bbox_2: 5.0, loss_bbox_3: 5.0, loss_bbox_4: 5.0, loss_bbox_5: 5.0, loss_bbox_6: 5.0, loss_bbox_7: 5.0, loss_bbox_8: 5.0, loss_bbox_dn: 5.0, loss_bbox_dn_0: 5.0, loss_bbox_dn_1: 5.0, loss_bbox_dn_2: 5.0, loss_bbox_dn_3: 5.0, loss_bbox_dn_4: 5.0, loss_bbox_dn_5: 5.0, loss_bbox_dn_6: 5.0, loss_bbox_dn_7: 5.0, loss_bbox_dn_8: 5.0, loss_bbox_interm: 5.0, loss_bbox_interm_0: 5.0, loss_bbox_interm_1: 5.0, loss_bbox_interm_2: 5.0, loss_bbox_interm_3: 5.0, loss_bbox_interm_4: 5.0, loss_bbox_interm_5: 5.0, loss_bbox_interm_6: 5.0, loss_bbox_interm_7: 5.0, loss_bbox_interm_8: 5.0, loss_bbox_interm_dn: 5.0, loss_bbox_interm_dn_0: 5.0, loss_bbox_interm_dn_1: 5.0, loss_bbox_interm_dn_2: 5.0, loss_bbox_interm_dn_3: 5.0, loss_bbox_interm_dn_4: 5.0, loss_bbox_interm_dn_5: 5.0, loss_bbox_interm_dn_6: 5.0, loss_bbox_interm_dn_7: 5.0, loss_bbox_interm_dn_8: 5.0, loss_ce: 4.0, loss_ce_0: 4.0, loss_ce_1: 4.0, loss_ce_2: 4.0, loss_ce_3: 4.0, loss_ce_4: 4.0, loss_ce_5: 4.0, loss_ce_6: 4.0, loss_ce_7: 4.0, loss_ce_8: 4.0, loss_ce_dn: 4.0, loss_ce_dn_0: 4.0, loss_ce_dn_1: 4.0, loss_ce_dn_2: 4.0, loss_ce_dn_3: 4.0, loss_ce_dn_4: 4.0, loss_ce_dn_5: 4.0, loss_ce_dn_6: 4.0, loss_ce_dn_7: 4.0, loss_ce_dn_8: 4.0, loss_ce_interm: 4.0, loss_ce_interm_0: 4.0, loss_ce_interm_1: 4.0, loss_ce_interm_2: 4.0, loss_ce_interm_3: 4.0, loss_ce_interm_4: 4.0, loss_ce_interm_5: 4.0, loss_ce_interm_6: 4.0, loss_ce_interm_7: 4.0, loss_ce_interm_8: 4.0, loss_ce_interm_dn: 4.0, loss_ce_interm_dn_0: 4.0, loss_ce_interm_dn_1: 4.0, loss_ce_interm_dn_2: 4.0, loss_ce_interm_dn_3: 4.0, loss_ce_interm_dn_4: 4.0, loss_ce_interm_dn_5: 4.0, loss_ce_interm_dn_6: 4.0, loss_ce_interm_dn_7: 4.0, loss_ce_interm_dn_8: 4.0, loss_dice: 5.0, loss_dice_0: 5.0, loss_dice_1: 5.0, loss_dice_2: 5.0, loss_dice_3: 5.0, loss_dice_4: 5.0, loss_dice_5: 5.0, loss_dice_6: 5.0, loss_dice_7: 5.0, loss_dice_8: 5.0, loss_dice_dn: 5.0, loss_dice_dn_0: 5.0, loss_dice_dn_1: 5.0, loss_dice_dn_2: 5.0, loss_dice_dn_3: 5.0, loss_dice_dn_4: 5.0, loss_dice_dn_5: 5.0, loss_dice_dn_6: 5.0, loss_dice_dn_7: 5.0, loss_dice_dn_8: 5.0, loss_dice_interm: 5.0, loss_dice_interm_0: 5.0, loss_dice_interm_1: 5.0, loss_dice_interm_2: 5.0, loss_dice_interm_3: 5.0, loss_dice_interm_4: 5.0, loss_dice_interm_5: 5.0, loss_dice_interm_6: 5.0, loss_dice_interm_7: 5.0, loss_dice_interm_8: 5.0, loss_dice_interm_dn: 5.0, loss_dice_interm_dn_0: 5.0, loss_dice_interm_dn_1: 5.0, loss_dice_interm_dn_2: 5.0, loss_dice_interm_dn_3: 5.0, loss_dice_interm_dn_4: 5.0, loss_dice_interm_dn_5: 5.0, loss_dice_interm_dn_6: 5.0, loss_dice_interm_dn_7: 5.0, loss_dice_interm_dn_8: 5.0, loss_giou: 2.0, loss_giou_0: 2.0, loss_giou_1: 2.0, loss_giou_2: 2.0, loss_giou_3: 2.0, loss_giou_4: 2.0, loss_giou_5: 2.0, loss_giou_6: 2.0, loss_giou_7: 2.0, loss_giou_8: 2.0, loss_giou_dn: 2.0, loss_giou_dn_0: 2.0, loss_giou_dn_1: 2.0, loss_giou_dn_2: 2.0, loss_giou_dn_3: 2.0, loss_giou_dn_4: 2.0, loss_giou_dn_5: 2.0, loss_giou_dn_6: 2.0, loss_giou_dn_7: 2.0, loss_giou_dn_8: 2.0, loss_giou_interm: 2.0, loss_giou_interm_0: 2.0, loss_giou_interm_1: 2.0, loss_giou_interm_2: 2.0, loss_giou_interm_3: 2.0, loss_giou_interm_4: 2.0, loss_giou_interm_5: 2.0, loss_giou_interm_6: 2.0, loss_giou_interm_7: 2.0, loss_giou_interm_8: 2.0, loss_giou_interm_dn: 2.0, loss_giou_interm_dn_0: 2.0, loss_giou_interm_dn_1: 2.0, loss_giou_interm_dn_2: 2.0, loss_giou_interm_dn_3: 2.0, loss_giou_interm_dn_4: 2.0, loss_giou_interm_dn_5: 2.0, loss_giou_interm_dn_6: 2.0, loss_giou_interm_dn_7: 2.0, loss_giou_interm_dn_8: 2.0, loss_mask: 5.0, loss_mask_0: 5.0, loss_mask_1: 5.0, loss_mask_2: 5.0, loss_mask_3: 5.0, loss_mask_4: 5.0, loss_mask_5: 5.0, loss_mask_6: 5.0, loss_mask_7: 5.0, loss_mask_8: 5.0, loss_mask_dn: 5.0, loss_mask_dn_0: 5.0, loss_mask_dn_1: 5.0, loss_mask_dn_2: 5.0, loss_mask_dn_3: 5.0, loss_mask_dn_4: 5.0, loss_mask_dn_5: 5.0, loss_mask_dn_6: 5.0, loss_mask_dn_7: 5.0, loss_mask_dn_8: 5.0, loss_mask_interm: 5.0, loss_mask_interm_0: 5.0, loss_mask_interm_1: 5.0, loss_mask_interm_2: 5.0, loss_mask_interm_3: 5.0, loss_mask_interm_4: 5.0, loss_mask_interm_5: 5.0, loss_mask_interm_6: 5.0, loss_mask_interm_7: 5.0, loss_mask_interm_8: 5.0, loss_mask_interm_dn: 5.0, loss_mask_interm_dn_0: 5.0, loss_mask_interm_dn_1: 5.0, loss_mask_interm_dn_2: 5.0, loss_mask_interm_dn_3: 5.0, loss_mask_interm_dn_4: 5.0, loss_mask_interm_dn_5: 5.0, loss_mask_interm_dn_6: 5.0, loss_mask_interm_dn_7: 5.0, loss_mask_interm_dn_8: 5.0} focus_on_box: false instance_on: true metadata: !!python/object/apply:detectron2.data.catalog.Metadata state: evaluator_type: coco image_root: D:\0_datasets\cow_stall\cam_vertical\aug_s json_file: D:\0_datasets\cow_stall\cam_vertical\aug_s\anot_aug_s.json name: cow_train thing_classes: [cowstay cow_lay] thing_colors:
• [0, 0, 255]
• [255, 0, 0] num_queries: 300 object_mask_threshold: 0.25 overlap_threshold: 0.8 pano_temp: 0.06 panoptic_on: false pixel_mean: [123.675, 116.28, 103.53] pixel_std: [58.395, 57.12, 57.375] sem_seg_head: target: projects.maskdino.modeling.MaskDINOHead ignore_value: -1 input_shape: res2: !!python/object:detectron2.layers.shape_spec.ShapeSpec {channels: 256, height: null, stride: 4, width: null} res3: !!python/object:detectron2.layers.shape_spec.ShapeSpec {channels: 512, height: null, stride: 8, width: null} res4: !!python/object:detectron2.layers.shape_spec.ShapeSpec {channels: 1024, height: null, stride: 16, width: null} res5: !!python/object:detectron2.layers.shape_spec.ShapeSpec {channels: 2048, height: null, stride: 32, width: null} loss_weight: 1.0 num_classes: 80 pixel_decoder: target: projects.maskdino.modeling.MaskDINOEncoder common_stride: 4 conv_dim: 256 feature_order: low2high input_shape: res2: !!python/object:detectron2.layers.shape_spec.ShapeSpec {channels: 256, height: null, stride: 4, width: null} res3: !!python/object:detectron2.layers.shape_spec.ShapeSpec {channels: 512, height: null, stride: 8, width: null} res4: !!python/object:detectron2.layers.shape_spec.ShapeSpec {channels: 1024, height: null, stride: 16, width: null} res5: !!python/object:detectron2.layers.shape_spec.ShapeSpec {channels: 2048, height: null, stride: 32, width: null} mask_dim: 256 norm: GN num_feature_levels: 3 total_num_feature_levels: 4 transformer_dim_feedforward: 2048 transformer_dropout: 0.0 transformer_enc_layers: 6 transformer_in_features: [res3, res4, res5] transformer_nheads: 8 transformer_predictor: {target: projects.maskdino.modeling.transformer_decoder.MaskDINODecoder, activation: relu, dec_layer_share: false, dec_layers: 9, dec_n_points: 4, dim_feedforward: 2048, dn: seg, dn_num: 100, dropout: 0.0, enforce_input_project: false, hidden_dim: 256, in_channels: 256, initial_pred: true, initialize_box_type: mask2box, learn_tgt: false, mask_classification: true, mask_dim: 256, nhead: 8, nheads: 8, noise_scale: 0.4, num_classes: '${..num_classes}', num_queries: 300, query_dim: 4, return_intermediate_dec: true, semantic_ce_loss: false, total_num_feature_levels: 4, two_stage: true} sem_seg_postprocess_before_inference: true semantic_on: false size_divisibility: 32 test_topk_per_image: 100 transform_eval: true optimizer: target: torch.optim.AdamW betas: [0.9, 0.999] lr: 0.0001 params: {target: detectron2.solver.get_default_optimizer_params, base_lr: '${..lr}', lr_factor_func: !!python/name:None.%3Clambda%3E '', weight_decay_norm: 0.0} weight_decay: 0.0001 train: amp: {enabled: false} checkpointer: {max_to_keep: 100, period: 13076} clip_grad: enabled: true params: {max_norm: 0.01, norm_type: 2} ddp: {broadcast_buffers: false, find_unused_parameters: false, fp16_compression: false} device: cuda eval_period: 3269 fast_dev_run: {enabled: false} init_checkpoint: C:\Users\Bumbi\PycharmProjects\mmdet\detrex\models\maskdino_r50_50ep.pth log_period: 50 max_iter: 65380 output_dir: ./output/model_cow_stay_lay_R50_25e4_b1024_180000k_aug_l1

[FengLi-ust]

That's interesting. I saw you also use initialize_box_type: mask2box to generate box from mask in query selection. It is expected that you box AP is higher than mask AP.

Is there anything special in your dataset?

[rbumbi]

Hello, thank you for your response. In my dataset aren't any special things, only images from stable (IP cams are placed on the ceiling), where I detect standing and lying cows. Masks in my dataset were created manually, and bboxes were automatically created from the masks; respectively, bboxes are precise. I expected higher box AP than mask AP, but my results were a little bit surprised me. Isn't anything bad in my config?

[FengLi-ust]

Not yet. You can wait for the result of more epochs. You can also check if your box output needs further post-processing.

[rbumbi]

Thank you for your advice, I will try it.

[rentainhe]

I'm closing this issue, feel free to reopen it if needed