biigle / maia

:m: BIIGLE module for the Machine Learning Assisted Image Annotation method
GNU General Public License v3.0
2 stars 3 forks source link

Improve novelty detection storage footprint with large clusters #166

Open mzur opened 3 months ago

mzur commented 3 months ago

If one image cluster of the novelty detection is very large (e.g. 4k images) the novelty detection could run out of disk space because it generates the (64 MB) novelty maps of each image first and then post-processes the maps. This can't really be fixed because it needs all novelty maps to determine the segmentation threshold and then the original maps to generate the actual segmentation. But maybe the novelty maps could be compressed to allow processing of larger image collections?