[Bug] Help on how to overlay segmentation dcm on image dcm

[b3r-prog]

Describe the Bug

We converted some input nifti files to dcm. We then run the files through a model and get the segmentation. We converted the segmentation as well into dcm. Using the ohif viewer we are able to view the original dcm files and converted segmentation dcm independently. We are trying to view both of them together but cant figure out how to do that. Is there a naming convention or a standard data loader that allows the viewer to load both and know what is a segmentation file and an original image loader.

Steps to Reproduce

1. Original image files in nifti format
2. Segmentation files of the original in nifti format

The current behavior

We can not view both at the same time

The expected behavior

A combined view of the original image and the segmentation

OS

mac

Node version

18.16

Browser

chrome

[fedorov]

We converted the segmentation as well into dcm.

Can you provide details of how you did that?

[b3r-prog]

@fedorov we just used a nifti to dcm converter library like this https://github.com/tomaroberts/nii2dcm. Our issue is overlaying the both dcms so the segmentation shows on the original image.

[fedorov]

DICOM Segmentation is a different kind of DICOM compared to CT/MR. You need specialized tools to do the conversion right. Here's one library that provides command line converter that you could use for that purpose: https://github.com/QIICR/dcmqi.

[b3r-prog]

Thanks for the reply. Is there a way to overlay the annotation once it is converted with dcmqi with the CT/MR dcm/nifti in OHIF viewer? I cant find any documentation

[fedorov]

You can overlay a properly constructed DICOM Segmentation object, if it is in the correct frame of reference. dcmqi is just one tool that you can use to create it.

If you want to see an example of DICOM SEG and how it looks in OHIF, here you go: https://viewer.imaging.datacommons.cancer.gov/v3/viewer/?StudyInstanceUIDs=1.2.840.113654.2.55.192012426995727721871016249335309434385&SeriesInstanceUIDs=1.2.840.113654.2.55.305538394446738410906709753576946604022,1.2.276.0.7230010.3.1.3.313263360.15787.1706310178.804490

You can download the corresponding files with the following:

pip install --upgrade idc-index
idc download 1.2.840.113654.2.55.305538394446738410906709753576946604022,1.2.276.0.7230010.3.1.3.313263360.15787.1706310178.804490

[b3r-prog]

@fedorov I downloaded and looked through the dcmqi library. I'm not sure it is applicable. I am trying to do the conversion to view (overlay the segmentation on the original dcm) in a cloud hosted ohif viewer. The goal is to do the conversion automatically.

Do you have an example where you did this conversion in ohif? The closest i could find is this itkimage2segimage. We run nifti through a model, get the output and are tring to overlay the output on the orignal image

[fedorov]

Yes, itkimage2segimage is the tool you need. OHIF does not do the conversion. It will display DICOM SEG in overlay, but you need to create it first.

[b3r-prog]

Just one additional question, if i have the nifti for both the original image and the segmentation, to ensure the converted files match and show together, what tool should i use for converting the original image nifti? (i.e. if i use itkimage2segimage to convert the nifti seg to SEG)

[fedorov]

You could start with nii2dcm tool you mentioned earlier. You may want to run dciodvfy DICOM validator https://dclunie.com/dicom3tools/dciodvfy.html on the output if you run into any issues downstream with SEG conversion or OHIF and that conversion result.

[JFKakaJFK]

I got this working using Slicer, although it is a bit cumbersome to match the segmentation to the volume. According to the documentation the way to do it is to:

1. Convert the nifti volume to dcm
2. Load that dcm and the nifti segmentation
3. Make the dcm the reference volume for the segmentation
4. Export the SEG

The resulting volume + segmentation dcm files load and show fine in OHIF (3.9.0-beta.66 with @ohif/extension-cornerstone-dicom-seg). I made this hacky script to automate the process (needs the QuantitativeReporting and SlicerRT extensions):

from typing import Optional

import datetime
import os

import slicer
from DICOMLib import DICOMUtils
import DICOMSegmentationPlugin

def nii2dcm(
    source_image_path: str,
    target_directory: str,
    source_segmentation_path: Optional[str] = None,
    **kwargs,
):
    """
    Converts a nifti image (optionally with segmentation) to DICOM

    !! Requires the QuantitativeReporting and SlicerRT extensions in the Slicer GUI !!

    This follows the procedure outlined in
    https://slicer.readthedocs.io/en/latest/user_guide/modules/segmentations.html#dicom-export
    and https://discourse.slicer.org/t/create-dicom-series-using-python/22871/3,
    i.e. first the nifti volume is converted to DICOM and reimported
    before the segmentation is added and exported to DICOM.
    """

    kwargs = {
        k: v.strftime("%Y%m%d") if isinstance(v, datetime.date) else v
        for k, v in kwargs.items()
    }

    volume_node = slicer.util.loadVolume(source_image_path)

    os.makedirs(target_directory, exist_ok=True)

    # https://slicer.readthedocs.io/en/latest/user_guide/modules/createdicomseries.html
    createdicomseries = slicer.modules.createdicomseries
    parameters = dict(
        inputVolume=volume_node,
        dicomDirectory=str(target_directory),
        patientID=kwargs.get("PatientID"),
        patientBirthDate=kwargs.get("PatientBirthDate"),
        patientSex=kwargs.get("PatientSex"),
        patientComments=kwargs.get("PatientComments"),
        studyDate=kwargs.get("StudyDate"),
        studyComments=kwargs.get("StudyComments"),
        studyDescription=kwargs.get("StudyDescription"),
        seriesDescription=kwargs.get("SeriesDescription"),
        seriesDate=kwargs.get("StudyDate"),
        contentDate=kwargs.get("StudyDate"),
    )
    cli_node = slicer.cli.runSync(
        createdicomseries, None, {k: v for k, v in parameters.items() if v is not None}
    )

    if cli_node.GetStatus() & cli_node.ErrorsMask:
        msg = cli_node.GetErrorText()
        slicer.mrmlScene.RemoveNode(cli_node)
        raise ValueError("CLI execution failed: " + msg)
    slicer.mrmlScene.RemoveNode(cli_node)

    if source_segmentation_path is not None:
        reference_volume = None
        with DICOMUtils.TemporaryDICOMDatabase() as db:
            DICOMUtils.importDicom(target_directory, db)
            patient_uids = db.patients()
            for patient_uid in patient_uids:
                node_ids = DICOMUtils.loadPatientByUID(patient_uid)
                for node_id in node_ids:
                    node = slicer.mrmlScene.GetNodeByID(node_id)
                    if not node or not node.IsA("vtkMRMLScalarVolumeNode"):
                        continue

                    reference_volume = node
                    break

                if reference_volume is not None:
                    break

            segmentation_node = slicer.util.loadSegmentation(source_segmentation_path)
            segmentation_node.SetReferenceImageGeometryParameterFromVolumeNode(
                reference_volume
            )

            sh_node = slicer.vtkMRMLSubjectHierarchyNode.GetSubjectHierarchyNode(
                slicer.mrmlScene
            )

            reference_volume_item = sh_node.GetItemByDataNode(reference_volume)
            study_item = sh_node.GetItemParent(reference_volume_item)
            segmentation_item = sh_node.GetItemByDataNode(segmentation_node)
            sh_node.SetItemParent(segmentation_item, study_item)

            exporter = DICOMSegmentationPlugin.DICOMSegmentationPluginClass()
            exportables = exporter.examineForExport(segmentation_item)
            for exp in exportables:
                exp.directory = target_directory

                for k in [
                    "PatientID",
                    "PatientBirthDate",
                    "PatientSex",
                    "PatientComments",
                    "StudyDate",
                    "StudyComments",
                    "StudyDescription",
                    "SeriesDescription",
                    "StudyDate",
                    "StudyDate",
                ]:
                    if k in kwargs:
                        exp.setTag(k, kwargs[k])

            exporter.export(exportables)

    # finally clear the scene
    slicer.mrmlScene.Clear(0)

if __name__ == "__main__":
    # https://slicer.readthedocs.io/en/latest/developer_guide/python_faq.html#what-is-the-python-console
    # <path to Slicer executable> --no-main-window --python-script nii2dcm.py
    nifti_volume = "<path to nifti image>"
    nifti_segmentation = "<path to nifti segmentation>"

    nii2dcm(
        source_image_path=nifti_volume,
        source_segmentation_path=nifti_segmentation,
        target_directory="<path to dcm>",
    )

    # Slicer window does not close automatically
    exit()