search

biigle / maia

:m: BIIGLE module for the Machine Learning Assisted Image Annotation method
GNU General Public License v3.0
2 stars 3 forks source link

Method to speed up manual review of autoencoder proposals #1

Closed mzur closed 5 years ago

mzur commented 6 years ago

This is the idea of a method to speed up the Largo-like manual review process of autoencoder proposals in the object detection workflow.

  1. For each proposal "patch", generate a simple feature vector, like a histogram. This can be used to compute a distance between any pair of patches.

  2. Determine a random patch P.

  3. Determine the x (%) of patches with the lowest distance to P. All these patches, including P, are a "batch". The idea is that most of the patches in a batch should be similar, e.g. showing the same object class. This makes it easy for the user to quickly label large portions of the batch as "interesting" or "uninteresting".

  4. Show the patches of the batch to the user in a Largo-like view. The user has to select either all interesting or all uninteresting patches (whichever is more frequent). The user must be able to select multiple consecutive patches in a single action.

  5. When the user finished labelling all patches of a batch, the results are sent back to the backend. Now the next batch is determined in the following way, which is intended to maximize the "information gain" from the manual labels: Determine the mean of all finished batches (in feature space, this can be done with an incremental update in each step). Then select the patch with the largest distance to the mean. This is the new patch P that is used to determine the next batch. Patches that belong to finished batches are not considered.

  6. Repeat until some stop criterion is statisfied. We don't know what to use here, yet. It should be something like "enough data collected so the FCN can be trained well for each object class".

mzur commented 6 years ago

An alternative idea: Perform a PCA (or t-SNE) on the patches in feature space and display a scatter plot of the first two PCs to the user. The user then can manually select a batch from this plot. All finished batches are removed from the plot. The rest of the method stays the same.

dlangenk commented 6 years ago

This would only work if you a) see clusters and b) the are meaningful. Most of the time you see nothing in the 2D view

mzur commented 5 years ago

I'll stick to the method described in the MAIA paper for now.