search

Lee-Gihun / MEDIAR

(NeurIPS 2022 CellSeg Challenge - 1st Winner) Open source code for "MEDIAR: Harmony of Data-Centric and Model-Centric for Multi-Modality Microscopy"
MIT License
137 stars 30 forks source link

current Mediar weights #19

Closed marius10p closed 5 months ago

marius10p commented 5 months ago

This is a small thing, but I was wondering how the current Mediar weights differ from the ones submitted to the challenge, if at all. We ran Mediar on the tuning set of the challenge using the evaluation code, and obtained a median F1 score of 0.9126 with TTA and ensembling (this json). Without TTA and ensembling, the median F1 score was 0.9091 (this json).

In the Mediar paper you report 0.9067, which nearly matches the largest score on the leaderboard of 0.9070.

Thanks.

Lee-Gihun commented 5 months ago

Oh, we hadn't even noticed that! Regarding your question, the current MEDIAR weights are exactly the same as the ones submitted to the challenge.

However, for the version of the prediction submitted, we limited the overlap size between patches and excluded TTA and ensembling for some images due to the time efficiency metric of the challenge evaluation. This was to ensure the running time met the time constraints. I think this might be the reason the score differs.

marius10p commented 5 months ago

I see, that makes sense, thanks so much for explaining.