probability map semantics

[udion]

the output of the command for generating probability map, like

python NCRF/wsi/bin/probs_map.py /WSI_PATH/Test_012.tif /CKPT_PATH/best.ckpt /CFG_PATH/cfg.json /MASK_PATH/Test_012.npy /PROBS_MAP_PATH/Test_012.npy

generates numpy file for the probability map, but the semantics of numpy file are bit odd, how do I intereprete the values?

for example, in the above case, If I print the max and min values of test_012.npy, then I get 0.05732796713709831 and 0.0 Which seems odd to me, as I was expecting max probability to be 1 (or close to one), but such low value represents that at every pixel probability of detecting cancer is so small?

[yil8]

@udion as I answered in the other issue, can you please check the level dimensions of your Test_026.tif is consistent with mine?

[udion]

@yil8, @sketchplanet No, the dimensions are not matching to what you've posted. In the following, I have tried another experiment and I am posting the output here.

I tried running tumor_001.tif PART OF TRAINING SET, hoping to see some outputs which I can make sense of

so, the level6 image looks like:

the corresponding tissue mask looks like:

the corresponding probability map (output of the network) looks like: Notice how the maximum value in the probability map is so small

Can you please tell me how did you get ground truth value to the plot? (as you have shown in README for the testing section for test_0026.tif figure b)

I tried converting the corresponding tumor_001.xml file to tumor_001.json and then visualizing the json file, the output of which looks like:

[yil8]

@udion Again, as I mentioned this issue #6 , Unless the level_dimension mismatch is fixed, I don't think any downstream analysis is meaningful. As for your tumor_001.tif, it's weird that there is a significant white margin in the bottom of your figure, which is not the case in my plot. I suspect this may be one potential issue? Did you use any padding? My tumor_001.tif plot at level 6 looks like this, notice there is no such white margin in the bottom

As for your tissue mask, it's dimension looks like around (3000, 7000), while my tissue mask dimension at level 6 is (1528, 3456). As for tumor_001.xml/json, it's coordinates are around (72000, 130000), but your plot has a dimension of around (4200, 2300). Not sure how you made the plot?

[udion]

@yil8 yes I had noticed the white boundary on the bottom and right side of many levels (not only on level 6), my hunch was also the same that these boundaries are causing problem. But, I don't know why I got them in first place.

[udion]

@yil8 the ground truth available for evaluation on google drive, looks something like: (for tumor_031 )

I can't find such things for test images, so I assume ground truth has to be created using annotation xml files. I am unable to install ASAP, I see you're converting xml to json, visualization of json will be something like shown above in the thread (with orange contours). Does it mean everything inside contour is labeled as cancer?

and so to get the ground truth, all the pixels inside the contour are set to 1 and 0 outside?

[yil8]

@udion Sadly, the camelyon16 organizers seem to have remove the ground truth masks in tif format. There used to be a set of Test_XXX_Mask.tif files that only the tumor pixels at level 0 are white([255, 255, 255]) and all the other region pixels are black ([0, 0, 0]). The evaluation code from the camelyon16 organizers still depends on these mask tif files (I highly encourage you to read the evaluation code, which is not something straight forward like binary classification accuracies). Unfortunately, I was trying to install ASAP and failed as well. So, I actually don't know how you can evaluate the FROC scores without using ASAP to generate the masks currently...But the contours do seem plausible that pixels inside the contours are tumor regions.

[udion]

@yil8

I wrote some code to take as input the json file and the level, and generate mask file at that level

example for test_001.tif mask at level 6:

would you be interested in accepting a pull request?

[yil8]

@udion Wow, nice! I'm definitely interested in taking a look. Thanks a lot!

[yux94]

@yil8 If it is convenient to you, could you please provide the download link? Thx~

[BeHappy1]

hi, I meet the same problem with you ,my probability value is also small . Could you tell me how you solve it.thank you.

[yil8]

@yux94 sorry, what download link are you referring to?

[yil8]

@BeHappy1 I have attached the tissue mask and the probability map of Test_026 at level 6 here for comparisons with yours. If everything goes smooth, you should obtain identical numerical results, which has been confirmed by one other user. Please let me know if you run into any further questions. Test_026_tissue_mask.npy Test_026_probs_map.npy

[yux94]

@yil8 sorry,i didn't make it clear. i mean the mask file with the type of '.tif' in the last step when you testing, since i can't find it on the web now, much appreciate!

[yil8]

@yux94 I don't have the download link for the mask files either....Maybe I should think about upload those mask files to Google drive or somewhere. But I guess that may violate the policy of camelyon16 organizers

[yux94]

@yil8 yes, i see, thx anyway~

[udion]

@BeHappy1

Check your system openslide version and confirm that it's 3.4.1

[yil8]

@udion Hi, if you did not run into any further issues of probability map semantics, I'm planing close this issue. Thanks

[usamabaig1]

@yil8

I wrote some code to take as input the json file and the level, and generate mask file at that level

example for test_001.tif mask at level 6:

would you be interested in accepting a pull request?

@udion Can you kindly share the code of converting xml to tiff ground truth files?

[yil8]

@usamabaig1 Cool, thanks!