search

OHIF / Viewers

OHIF zero-footprint DICOM viewer and oncology specific Lesion Tracker, plus shared extension packages
https://docs.ohif.org/
MIT License
3.12k stars 3.29k forks source link

handle fractional SEG objects #1346

Open pieper opened 4 years ago

pieper commented 4 years ago

Request

Related to #1345, the segmentation is encoded as fractional but we currently don't load it.

Instead we throw this as an error:

Uncaught (in promise) Error: Fractional segmentations are not supported

What feature or change would you like to see made?

We should probably just threshold this to binary for now. Better of course would be to handle the fractional case as a kind of heatmap overlay.

Why should we prioritize this feature? It's part of the DICOM standard and we have examples of data encoded this way.

JamesAPetts commented 4 years ago

Better of course would be to handle the fractional case as a kind of heatmap overlay.

Yeah this one is interesting, this would be the best thing to do, but it'd essentially be a different data type in cornerstoneTools. I think the simplest thing to do would be to save each segment as its own labelmap as per standard and give it its own colorLUT that is the same color with different alpha per value?

Is it common to have multiple fractional segments in one DICOM SEG on one referenced series?

pieper commented 4 years ago

Is it common to have multiple fractional segments in one DICOM SEG on one referenced series?

Not sure if there's any consensus on this, but if it's okay according to the standard then probably somebody will do it. It would be great if we could work on promoting best practices for how people should structure their SEG instances. (Comments @fedorov ?)

JamesAPetts commented 4 years ago

I'm sure its DICOM compliant, but the web dev in me is already screaming internally about the potential consequences on resource allocation :).

fedorov commented 4 years ago

From a quick look, I am not sure how this is related to #1345 - can you clarify? Also, considering we don't have any fractional SEGs in TCGA, and it's not in IDC MVP, I don't consider this to be IDC priority.

I have very limited experience with fractional SEG, I don't think I have seen any examples other than the objects produced by BrainLab, and I don't know of any tools that create fractional SEGs.

fedorov commented 4 years ago

We confirmed with @pieper that the segmentations in that kidney collections are binary, so the question is why that fractional error is coming up.

fedorov commented 2 years ago

It turns out we do have fractional segmentations in IDC, which I think turns this supporting fractional segmentations into a priority. @Punzo how difficult is this going to be to implement?

Example dataset from IDC: https://viewer.imaging.datacommons.cancer.gov/viewer/1.3.6.1.4.1.14519.5.2.1.7695.4164.330974290504454152904316943429

Punzo commented 2 years ago

@fedorov supporting fractional segmentations is rather complicated and it would involve various steps: 1) for cornerstone2D we need to modify the segmentation tools infrastructure to allow rendering each segment with a LUT. This will be very heavy in resources as James already pointed out: e.g. for each segment we would need to allocate its own label map (and give it its own colorLUT that is the same color with different alpha per value); 2) In dmcjs/OHIF load the fractional segmentation with the new cornerstone tools infrastructure. 3) apply the same concept for VTK extension. Here comes an issue: since in OHIFv2 we support only one labelMap per segmentation. We had already this issue with overlapping binary segmentations. In this last case, we decided to skip the support for overlapping binary segmentations in the VTK extension for OHIFv2 (the main issue is memory, since in the vtk extension you need to multiply by 3 the memory usage); 4) revisit UI (e.g. settings have been designed specifically for binary label maps); 5) probably makes sense to visualize only one series at time (while for binary label maps we allow rendering from different series, i.e. active/inactive label maps).

pieper commented 2 years ago

I believe handling fractional segmentation is going to be much easier in OHIF v3 (modulo resource issues). Maybe for v2 we could just binarize the segmentations (with a user warning).

fedorov commented 2 years ago

Maybe for v2 we could just binarize the segmentations (with a user warning).

This might make sense. It might be that all fractional segs are actually binary, but saved as fractional. We should to check.

Punzo commented 2 years ago

Ok I will check the actual values in the array. In case, should I use a simple thresholding for binarizing them? I fear anything more convoluted can be slow in JavaScript.

pieper commented 2 years ago

yes, just thresholding should be fine. Probably just zero/non-zero but let us know if that doesn't look right for the data values.

Punzo commented 2 years ago

PR with summary of my findings is here: https://github.com/dcmjs-org/dcmjs/pull/309 @pieper @fedorov

georg-walther commented 1 year ago

I would really appreciate OHIF's support for fractional segmentations. I have heatmaps formatted as DICOM-SEG. @fedorov There are tools to create fractional SEGs: https://github.com/razorx89/pydicom-seg/tree/feature/write-fractional

fedorov commented 1 year ago

@georg-walther the issue is that (from my perspective, as IDC https://portal.imaging.datacommons.cancer.gov/ lead), we do not have any fractional SEG series in IDC, while at the same time we have a lot of other issues in OHIF that are in the critical path for IDC, so I cannot prioritize working on this issue.

Can you at least share a sample fractional SEG and the accompanying image series, and attach to this issue, so this can be worked on in the future?