drivendataorg / concept-to-clinic

ALCF Concept to Clinic Challenge
https://concepttoclinic.drivendata.org/
MIT License
367 stars 146 forks source link

Added detection evaluation method for detection #301

Closed swarm-ai closed 6 years ago

swarm-ai commented 6 years ago

Description

Correct detection A detection can be treated as a correct detection if the intersection-over-union (IoU) of the ground truth and the detected bounding boxes is larger than a predefined threshold. The concept is shown as below. (Images are borrowed from https://www.pyimagesearch.com/2016/11/07/intersection-over-union-iou-for-object-detection/)

Reference to official issue

Issue #271

Motivation and Context

We extend the IoU calculation from 2D to 3D but in order to be simple, we only deal with 3D boxes / cubes. Although the detections we have are 3D spheres output from the grt123 system. The threshold we currently use is 0.5 but can easily be changed (‘correct_detection_threshold’ in evaluate_detection.py). We also did some experiments using different thresholds, please check the evaluation results below.

How Has This Been Tested?

We have tested the performance on the NSCLC Radiogenomics data set found here: http://www.cibl-harvard.org/data

You can see we include a results file that can be compared with the ground truth file in concept-to-clinic/prediction/src/algorithms/training/detector/label/custom_annos.csv

Screenshots (if appropriate):

screen shot 2018-01-25 at 3 57 17 pm

CLA

lamby commented 6 years ago

This is awesome :)