modAL-python / modAL

A modular active learning framework for Python
https://modAL-python.github.io/
MIT License
2.24k stars 324 forks source link

Faster way to get uncertainty scores on all data points upon each update of model? #143

Open ghost opened 3 years ago

ghost commented 3 years ago

Hi, I'm new to modAL and machine learning. This might be a dumb question but I didn't figure it out after I read through docs and reproduced some of the tutorials.

I have a binary text classification model, imdb reviews (positive or negative). I want to know the uncertainty score of each instances in the pool after query.

I'm using a for loop to loop through all elements in the tfidf_transformed matrix, and get classifier_uncertainty(learner, element).

r = [classifier_uncertainty(self.learner, i) for i in self.x_pool_vec]

there are more than 6000 elements in the matrix and each round of the iteration takes proximately 0.13..... secs.

I wonder is there any more efficient way I get get the things I need?

Thank you so much!

I can provide more code if needed.