Question regarding evaluation function (minor bug?)

DeepMicroscopy / MIDOGpp

A large, multi-domain mitotic Figure Dataset

MIT License

12 stars 8 forks source link

Question regarding evaluation function (minor bug?) #8

Closed FabianHoerst closed 5 months ago

FabianHoerst commented 5 months ago

Hello,

i think there might be a bug in the evaluation code or I might missunderstand something: In the evaluation function, first, the tuple is [x,y,z,label,score]. However, for creating the pred_dict, it is referenced as [x,y,z,score,label]. Why is the label used as score here? https://github.com/DeepMicroscopy/MIDOGpp/blob/c2726b1f40a1bfe4f0acce9c7f1d186eec12e914/evaluation.py#L86-L94

maubreville commented 5 months ago

Hey Fabian.

Thanks for that. We'll look into it.

Best,

Marc

FraukeWilm commented 5 months ago

Dear Fabian,

Yes, this was indeed a bug in our evaluation code which influenced the mAP calculation. Thank you for brining this to our attention. We have made the necessary code changes and will prepare an addendum for our manuscript. Fortunately, all evaluations in the manuscript were based on the F1 score, which was not affected by the bug.

Kind regards, Frauke

FabianHoerst commented 5 months ago

Thank you for the response!

Sorry for the problem it may have caused! Luckily, your mAP results should get even better with the change. Best, Fabian