Jun-02-2016

[azhe825]

Things done

1. Target Problem:
   • sampling bias --> imbalance problem in active learning (results delta can be more significant)
2. Existing literature on solving imbalance problem in active learning scenario
   • one paper suggests that active learning outperforms SMOTE on imbalanced data. Learning on the border: active learning in imbalanced data classification 2007
     • SVM only cares about support vectors
     • data near decision boundary are less imbalanced
   • Use human resources (filtering and searching) Why label when you can search?: alternatives to active learning for applying human resources to build classification models under extreme class imbalance 2010 (Guided Learning)
   • Boosted Disagreement with QBC: Reducing class imbalance during active learning for named entity annotation.
3. What existing literature lack
   • all methods focus on how to select training examples for next generation.
   • all methods assume that we already have a initially labeled training set. (Except for Hierarchical sampling for active learning. The problem for this method is that it totally abandoned the good nature of active learning.)
   • our assumptions:
     • 1. imbalance in initial training set will affect the active learning performance
     • 2. Hierarchical clustering can balance the initial training set
     • 3. For new stages, we need to consider expert knowledge. e.g. keyword search through elasticsearch first to retrieve a more balanced initial training set.
4. Negative Results (on multi-classification problem)

The entropy maximization methods do not make a single difference from random sampling!!!

To Do

1. reduce the problem to binary classification, target class is minority.
2. if still does not work out, consider keywork search.