leo-p
commented
7 years ago Very good introduction to active learning.
1. Scenarios
There are three mains scenari:
- Pool-based: a large amount of unlabeled data is available and we need to chose which one to annotate next.
- Stream-based: same as above except example come one after the other.
- Membership query synthesis: we can generate the point to label.
2. Query Strategy Frameworks
2.1. Uncertainty Sampling
Basically how to evaluate the informativeness of unlabeled instances and then select the most informative.
2.1.1. Least Confident
Query the instances about which the algorithm is least certain how to label.
Most used by discard information on all other labels.
2.1.2. Margin Sampling
Use the first two labels and chose the instance for which the different between the two is the smallest.
2.1.3. Entropy
Instead of using the two first labels, why not use all of them?
2.2. Query-By-Committee
A committee of different models is trained. They then vote on which instance to label and the one for which they most disagree is chosen.
To measure the level of disagreement, one can either use:
-
Vote entropy:
-
Kullback-Leibler divergence:
2.3. Expected Model Change
Selects the instance that would impart the greatest change to the current model if we knew its label.
- Expected Gradient Length: compute the gradient for all instances and find the one with the largest magnitude on average for all labels.
2.4. Expected Error Reduction
Measure not how much the model is likely to change, but how much its generalization error is likely to be reduced. Either by measuring:
-
Expected 0/1 loss: to reduce the expected total number of incorrect predictions. A new model needs to be trained for every label and instance, very greedy.
-
Expected Log-Loss: maximizing the expected information gain of the query. Still very greedy in computation! Not really usable except if the model can be analytically resolved instead of re-trained.
2.5. Variance Reduction
Reduce generalization error indirectly by minimizing the output variance.
2.6. Density-Weighted Methods
With the left function is the informativeness of x and the right function represents average similarity to all other instances in the input distribution
http://burrsettles.com/pub/settles.activelearning.pdf
The key idea behind active learning is that a machine learning algorithm can achieve greater accuracy with fewer training labels if it is allowed to choose the data from which it learns. An active learner may pose queries, usually in the form of unlabeled data instances to be labeled by an oracle (e.g., a human annotator). Active learning is well-motivated in many modern machine learning problems, where unlabeled data may be abundant or easily obtained, but labels are difficult, time-consuming, or expensive to obtain. This report provides a general introduction to active learning and a survey of the literature. This includes a discussion of the scenarios in which queries can be formulated, and an overview of the query strategy frameworks proposed in the literature to date. An analysis of the empirical and theoretical evidence for successful active learning, a summary of problem setting variants and practical issues, and a discussion of related topics in machine learning research are also presented.