search

biigle / maia

:m: BIIGLE module for the Machine Learning Assisted Image Annotation method
GNU General Public License v3.0
2 stars 3 forks source link

Performance evaluation #112

Open mzur opened 1 year ago

mzur commented 1 year ago

We could offer a simple method to evaluate the detection performance of MAIA. Users can choose to separate a "test set" from the training annotations (if they use existing annotations or UnKnoT). This is a percentage of the total annotations (e.g. 5%). Behind the scenes, whole images are successively excluded from the training set until around 5% of the annotations are excluded (try to balance this so each label has more or less the same number of excluded annotations).

When the object detector is trained, test it on the test set and report the recall and precision. This can also be displayed in an ECharts visualization.

mzur commented 9 months ago

Alternatively, users can be asked to start annotating a few (5, 10, 20?) images of the volume while the MAIA job is running. These annotations could be automatically used to test the performance of the detection model.