How does an active learning loop work in real life?

[vahuja4]

I have gone through the notebooks and am very keen to get started, but I would like to know how does an active learning loop work in the real-world. Every iteration requires a few samples to be labelled and then a new model has to be trained. How do we manage the versioning etc? How to make the process streamlined?

[chschroeder]

Hi @vahuja4,

for a quick start with practical active learning I would recommend the argilla tutorial.

Compared to the examples which show automated active experiments, you just need to adapt the code to 1. obtain the labels from user input and 2. update your train set with the given labels. I have already plans for such an example, which might come soon.

How do we manage the versioning etc?

I am unsure if the "versioning" refers to models or data. For the former, you can take a look at the documentation which shows how the current active learner can be saved (This is also possible for the underlying model which I also need to add to the documentation.)

Regarding the data, how would a versioning look like? You are probably thinking of a "history" which shows changes in the assigned labels, right? This would be something that downstream applications need to provide. For small-text, the goal is to provide the core algorithms that you can build applications with exactly such features on top of it.