keiserlab / plaquebox-paper

Repo for Tang et al, bioRxiv 454793 (2018)
MIT License
36 stars 25 forks source link

Unable to reproduce CNN scores #2

Open ciaran-coleman opened 2 years ago

ciaran-coleman commented 2 years ago

Hello,

I am attempting to reproduce your results but am seeing differences in the WSI CNN count results. Example WSI NA5001_2AB:

All packages I have matched, e.g. version 3.4.1 of libopencv/opencv/py-opencv, version 0.3.0 of torch, pyvips 2.1.2 with libvips 8.2.2. I should say that I am on a Windows 10 machine so this could very well be the problem, but just wanted to check first that I haven't missed something else.

When running notebook 2.2) I get the same AUC-PRC scores and so it seems the model is working correctly and giving the correct confidence.

I have also checked the stain normalization is working as intended by comparing 256x256 patches from the training data set provided in the Zenodo link to the equivalent cropped images produced when processing the WSIs with notebooks 1.1)-1.3). So I believe the WSIs are correctly stain normalized.

Would it be at all possible to see your prediction confidence heatmap for the example above so that I can inspect? I suspect this is where the differences lie that lead to the discrepancy.