Automatic classification to mitigate class imbalance

[mzur]

Training of Mask R-CNN in the current implementation can suffer from class imbalance. As all selected training proposals are treated as the same class, objects that are more abundant than others will be detected more accurately during instance segmentation. To apply a class imbalance mitigation strategy, the training proposals need class information.

1. The class information could be provided manually by not only selecting an interesting training proposal but also by attaching an appropriate label to it. This will cosiderably slow down the whole process of annotating with MAIA, however. This needs a thorough evaluation of the improved detection performance and the worsened annotation speed.

2. Class information could be obtained automatically. The selected training proposals could be clustered and each cluster be treated as some arbitrary class. This class could be used to apply a class imbalance mitigation strategy during training. The effectiveness of this approach needs to be evaluated, too.

I think 2. could be a nice student project.

[mzur]

If UnKnoT is used, the training annotations (probably) already have labels attached. The labels can be used to use a class imbalance mitigation strategy for training.