Object Detection not learning some classes

I have been training the object detection API on my own dataset of PASCAL VOC 2012 format. After 5 days and 8k+ steps of training, the model was able to learn all of my desired classes with exception to one: resetButton.

A glimpse into the produced graphs, the mAP for resetButton lifts off but then drastically falls back down and keeps exhibiting this pattern onwards. The training was done from scratch without any pre-loaded models with faster_rcnn_resnet152 as the feature_extractor.

My loss function has been fluctuating between 0.2-1.6 and the reason I'm still even letting it run is hoping that the resetButton mAP will finally lift off. At this point I feel like I'm over-fitting all of my other categories as their respective mAP have already converged.

Any reason why this is happening?

Precision First Graph Precision Second Graph Precision Third Graph Precision Fourth Graph General Precision Graph

Pipeline Configuration

# Faster R-CNN with Resnet-152 (v1), configured for Oxford-IIIT Pets Dataset.
# Users should configure the fine_tune_checkpoint field in the train config as
# well as the label_map_path and input_path fields in the train_input_reader and
# eval_input_reader. Search for "PATH_TO_BE_CONFIGURED" to find the fields that
# should be configured.

model {
  faster_rcnn {
    num_classes: 8
    image_resizer {
      keep_aspect_ratio_resizer {
        min_dimension: 600
        max_dimension: 1024
      }
    }
    feature_extractor {
      type: 'faster_rcnn_resnet152'
      first_stage_features_stride: 16
    }
    first_stage_anchor_generator {
      grid_anchor_generator {
        scales: [0.25, 0.5, 1.0, 2.0]
        aspect_ratios: [0.5, 1.0, 2.0]
        height_stride: 16
        width_stride: 16
      }
    }
    first_stage_box_predictor_conv_hyperparams {
      op: CONV
      regularizer {
        l2_regularizer {
          weight: 0.0
        }
      }
      initializer {
        truncated_normal_initializer {
          stddev: 0.01
        }
      }
    }
    first_stage_nms_score_threshold: 0.0
    first_stage_nms_iou_threshold: 0.7
    first_stage_max_proposals: 300
    first_stage_localization_loss_weight: 2.0
    first_stage_objectness_loss_weight: 1.0
    initial_crop_size: 14
    maxpool_kernel_size: 2
    maxpool_stride: 2
    second_stage_box_predictor {
      mask_rcnn_box_predictor {
        use_dropout: false
        dropout_keep_probability: 1.0
        fc_hyperparams {
          op: FC
          regularizer {
            l2_regularizer {
              weight: 0.0
            }
          }
          initializer {
            variance_scaling_initializer {
              factor: 1.0
              uniform: true
              mode: FAN_AVG
            }
          }
        }
      }
    }
    second_stage_post_processing {
      batch_non_max_suppression {
        score_threshold: 0.0
        iou_threshold: 0.6
        max_detections_per_class: 5
        max_total_detections: 50
      }
      score_converter: SOFTMAX
    }
    second_stage_localization_loss_weight: 2.0
    second_stage_classification_loss_weight: 1.0
  }
}

train_config: {
  batch_size: 1
  optimizer {
    momentum_optimizer: {
      learning_rate: {
        manual_step_learning_rate {
          initial_learning_rate: 0.0003
          schedule {
            step: 0
            learning_rate: .0003
          }
          schedule {
            step: 900000
            learning_rate: .00003
          }
          schedule {
            step: 1200000
            learning_rate: .000003
          }
        }
      }
      momentum_optimizer_value: 0.9
    }
    use_moving_average: false
  }
  gradient_clipping_by_norm: 10.0
  data_augmentation_options {
    random_horizontal_flip {
    }
  }
}

train_input_reader: {
  tf_record_input_reader {
    input_path: "****/pascal_train.record"
  }
  label_map_path: "****/mine_label_map.pbtxt"
}

eval_config: {
  num_examples: 123
}

eval_input_reader: {
  tf_record_input_reader {
    input_path: "****/pascal_val.record"
  }
  label_map_path: "****/mine_label_map.pbtxt"
  shuffle: false
  num_readers: 1
}

tensorflow / models

Object Detection not learning some classes #2027