I am not able to detect 1000's of instances of object in single image by using Mask-RCNN model.

[omi9y]

I have trained Mask-RCNN model with dataset in which in every single image, I am having 800-1000 instances of object which are small but detectable. And I am to train the model successfully.

But when I am using this model to test on same set of images, I am only able to detect the instance of object from any one side till 3/4th or 1/2 of image. But not all the instances of object.

What I feel that my model is able to detect the object but it is failing to detect all the instance. Kindly help on this.

Following is my config file and I have changed the path for labelmap.pbtxt and dataset path as per requirement. I have added few parameters such as max_number_of_boxes, max_num_boxes_to_visualize and tuned it to upper limits value. I have also tuned max_detections_per_class, max_total_detections and first_stage_max_proposals parameters to its maximum value after which incrementing these values throws memory error.

model { faster_rcnn { number_of_stages: 3 num_classes: 1 image_resizer { fixed_shape_resizer { height: 1024 width: 1024 } } feature_extractor { type: 'faster_rcnn_inception_resnet_v2_keras' } first_stage_anchor_generator { grid_anchor_generator { scales: [0.25, 0.5, 1.0, 2.0] aspect_ratios: [0.5, 1.0, 2.0] height_stride: 16 width_stride: 16 } } first_stage_box_predictor_conv_hyperparams { op: CONV regularizer { l2_regularizer { weight: 0.0 } } initializer { truncated_normal_initializer { stddev: 0.01 } } } first_stage_nms_score_threshold: 0.0 first_stage_nms_iou_threshold: 0.7 first_stage_max_proposals: 1000 first_stage_localization_loss_weight: 2.0 first_stage_objectness_loss_weight: 1.0 initial_crop_size: 17 maxpool_kernel_size: 1 maxpool_stride: 1 second_stage_box_predictor { mask_rcnn_box_predictor { use_dropout: false dropout_keep_probability: 1.0 fc_hyperparams { op: FC regularizer { l2_regularizer { weight: 0.0 } } initializer { variance_scaling_initializer { factor: 1.0 uniform: true mode: FAN_AVG } } } mask_height: 33 mask_width: 33 mask_prediction_conv_depth: 0 mask_prediction_num_conv_layers: 2 conv_hyperparams { op: CONV regularizer { l2_regularizer { weight: 0.0 } } initializer { truncated_normal_initializer { stddev: 0.01 } } } predict_instance_masks: true } } second_stage_post_processing { batch_non_max_suppression { score_threshold: 0.0 iou_threshold: 0.6 max_detections_per_class: 350 max_total_detections: 350 } score_converter: SOFTMAX } second_stage_localization_loss_weight: 2.0 second_stage_classification_loss_weight: 1.0 second_stage_mask_prediction_loss_weight: 4.0 resize_masks: false } }

train_config: { batch_size: 1 max_number_of_boxes: 200 num_steps: 5000 optimizer { momentum_optimizer: { learning_rate: { cosine_decay_learning_rate { learning_rate_base: 0.008 total_steps: 5000 warmup_learning_rate: 0.0 warmup_steps: 500 } } momentum_optimizer_value: 0.9 } use_moving_average: false } gradient_clipping_by_norm: 10.0 fine_tune_checkpoint_version: V2 fine_tune_checkpoint: "PATH_TO_BE_CONFIGURED" fine_tune_checkpoint_type: "detection" use_bfloat16: false data_augmentation_options { random_horizontal_flip { } } }

train_input_reader: { label_map_path: "PATH_TO_BE_CONFIGURED" max_number_of_boxes: 200 tf_record_input_reader { input_path: "PATH_TO_BE_CONFIGURED" } load_instance_masks: true mask_type: PNG_MASKS }

eval_config: { metrics_set: "coco_mask_metrics" eval_instance_masks: true use_moving_averages: false batch_size: 1 include_metrics_per_category: false max_num_boxes_to_visualize: 200 }

eval_input_reader: { label_map_path: "PATH_TO_BE_CONFIGURED" max_number_of_boxes: 200 shuffle: false num_epochs: 1 tf_record_input_reader { input_path: "PATH_TO_BE_CONFIGURED" } load_instance_masks: true mask_type: PNG_MASKS }

[omi9y]

Are there any updates on this issue because its already more than a month now @jaeyounkim @tombstone @jch1 @pkulzc @saikumarchalla

[0rhisia0]

You need to change the max_number_of_boxes parameter in your config to something suitable (for your case maybe 1200, it is currently set at 200). Also consider increasing the first_stage_max_proposals value to around 2000. If memory is the issue as you have indicated, there is not really any work around except tiling your images, and running the detector on each image.

[omi9y]

Hi @0rhisia0,

I tried those but it starts giving me out of memory error.

[Petros626]

Hi @0rhisia0,

I tried those but it starts giving me out of memory error.

Did you retrain the model with the changed mask number?