search

IMSY-DKFZ / htc

Semantic organ segmentation for hyperspectral images.
Other
25 stars 5 forks source link

[Question] Full semantic annotations #24

Open alfieroddan opened 6 months ago

alfieroddan commented 6 months ago

:question: Question

Is it possible to access the semantic annotations?

Description

Hi there just a quick question about the annotations.

From what I gather the full dataset we are able to download from the website only contains binary segmentation annotations (not semantic) like the below:

Screenshot 2024-01-09 at 17 50 01

As opposed to the semantic annotations see in Figure 8 of Robust deep learning-based semantic organ segmentation in Hyperspectral images.

Currently if we train using the full dataset and your framework the model is learning some organs as a background class and also its own class. Is this the intended optimisation or is the full annotation not released?

Once again thank you very much for the work on this package, I have been enjoying getting to grips to it and feel as if I'm almost at the point of contribution.

JanSellner commented 6 months ago

You are right, the HeiPorSPECTRAL dataset only contains polygon annotations with (usually) one organ in an image.

In our MIA paper, we used fully semantically annotated images which are not open source and the corresponding images are also from different subjects (no overlap with the HeiPorSPECTRAL dataset).

Are you using one of our pretrained models and then finetune on the HeiPorSPECTRAL dataset? The classes do not overlap completely, e.g., there is no background class in the HeiPorSPECTRAL dataset, so some confusions may be expected. During finetuning, I could also imagine that the network forgets some classes (like background) if it does not see them anymore. Early stopping may be helpful here. Alternatively, you could also first make a prediction on all images from the HeiPorSPECTRAL dataset using our pretrained image model and use the predictions as new semantic labels (with the actual labels from the HeiPorSPECTRAL superimposed of course).

Thanks again for the interest in our work and I hope that our models and the dataset are still useful for you even if the semantic dataset isn't open source.