Develop a better laser point detection algorithm

[mzur]

We collected the following usage statistics from biigle.de:

Laser point detection was attempted on 159368 images. Of these detections, 103059 (65 %) were successful. Of the successful cases, 23423 (23 %) were automatic detections. This means that only 15 % of the attempted laser point detections were successful automatic detections.

If we assume that users prefer automatic detections and only annotate manually if the detection fails, we could conclude that the laser point detection algorithm (as implemented here) does not work very well. There could be a need for a better and carefully tested algorithm that can replace DELPHI.

[mzur]

It's probably possible to train a deep learning laser point detection model based on the data we have gathered in BIIGLE. This could be investigated in a student project.

[mzur]

We want to start working on this now and replace Delphi with a DL-based approach. A few notes:

• The detection for a whole volume is initiated in the ProcessVolumeDelphiJob. it first processes all images with manual laser point annotations. This should work the same later. Then it executes the "gather" step of Delphi with a subset of annotated images. Finally, it submits a ProcessDelphiJob for all images that have no manually annotated laser points.

• The jobs use the DelphiGather and DelphiApply support classes to execute Python scripts.

• The Python scripts are delphi_gather.py, delphi_gather_finish.py and delphi_apply.py.

• The ProcessImageDelphiJob does the Delphi detection on a single image.

All these places have to be updated for the new DL-based approach. Instead of running one job for each image on the CPU, a single job for all (or maybe a chunk of 1000) images on the GPU may be more efficient. This is because loading a DL model on the GPU takes time and once the model is loaded, it is much faster to process a batch of files already lying around on the local disk.