thomashopkins32 / HuBMAP

Hacking the Human Vasculature (Kaggle Competition)
Apache License 2.0
0 stars 0 forks source link

Implement metrics for evaluation #3

Closed thomashopkins32 closed 1 year ago

thomashopkins32 commented 1 year ago

Look into different metrics for evaluating image segmentation problems. The most common one I can think of (which will also be used by Kaggle for scoring the competition) is IoU (Intersection over Union).

We may need custom implementations for this but we also might be able to use the COCO package to do this work.

thomashopkins32 commented 1 year ago

It looks like the image segmentation metric for the competition uses a separate evaluation script that operates on the encoded mask. We can add this to our repo to evaluate our results prior to submission but re-implementing this for something like cross-validation might be too expensive.

thomashopkins32 commented 1 year ago

Actually it seems to be mAP at an IoU thresholds > 0.6. This should be available in some package.

thomashopkins32 commented 1 year ago

It seems like pycocotools has the best implementation to use for this.