search

modAL-python / modAL

A modular active learning framework for Python
https://modAL-python.github.io/
MIT License
2.2k stars 324 forks source link

Which sampling method is best for very unbalanced data? #162

Open viandres opened 1 year ago

viandres commented 1 year ago

Hi!

I am wondering, which of the implemented sampling strategies handles unbalanced data best? I believe if I get the top 10000 uncertain data instances, but 99 % are in the same class, this would not help much for the next training process iteration, right?

Thank you in advance!

TomKingsfordUoA commented 1 year ago

For unbalanced data, where the estimator hasn't been trained on minority classes, typically the uncertainty measure fails to give epistemic uncertainty so won't (necessarily) sample the minority classes. Unlike uncertainty-based active learning, diversity-based AL handles this well. I've produced some diversity-based implementations privately and will look to submit a PR in the near future.