Establish recommendations/features when SEG and image geometries are very different

[fedorov]

Triggered by this use case: https://projectweek.na-mic.org/PW40_2024_GranCanaria/Projects/LinkingSegmentationAndImagingDataWithDifferentGeometriesUsingDcmqi/

Currently, references are established at the individual segment frame level, and referenced instances are then included in the ReferencedSeriesSequence > ReferencedInstanceSequence. This approach will not be practical when the source has a different geometry, since establishing correspondence between segmentation frames and source images will be either not practical (we we ask users to define such correspondence while parameterizing conversion) or under-defined and complex (would it be fair to assume that any slice intersecting the SEG frame should be referenced? if we have any significant number of frames, the size of these references will be prohibitive).

Ideally, it would be nice to identify a mechanism where we could at the series level reference the segmented image series in SEG. @dclunie is there any way to do this?

[fedorov]

Following on the comment from @dclunie in the aforementioned page ...

Perhaps consider something like recent RT Structure Set addition to standard that added SourceSeriesInformationSequence and SourceSeriesSequence (CP 2296 Provide additional ROI parameters to avoid parsing strings)

David, this looks exactly like what we need!

When you say "perhaps consider", are you suggesting we can just start using that sequence, or that we would need to have a CP to add it to SEG?

[dclunie]

When you say "perhaps consider", are you suggesting we can just start using that sequence, or that we would need to have a CP to add it to SEG?

It was added only for RT Structure Sets, since RT planning was the use case.

To use it in SEG would require a new CP, and the addition would need to be reworded to explain what it "means" in a SEG usage context.

[fedorov]

@dclunie here is the rationale for this addition. Do you need anything to proceed with the CP?

Rationale: Explicit reference of the series that was used to generate SEG is important for a variety of reasons. A typical DICOM studies contains multiple series, but only a subset of those may be used for performing segmentation. These could be series that have different contrast or acquisition protocol. There could also be motion or deformation of the anatomy being segmented across different series. It is important to communicate what series were used for segmentation for the purposes of provenance and reproducibility.

Limitations of the current SEG: Currently, it is impossible to reference the series that was used to create SEG directly. Instead, it is possible to reference individual instances that were used for creating a specific SEG frame in the per-frame functional groups sequence. This approach is problematic for several reasons. First, it requires establishing frame-level references, which may be difficult or impossible (e.g., in the situation where orientation of the SEG frames is orthogonal to that of the image slices). Second, this may lead to prohibitive size of the PerFrameFunctionalGroups sequence with the large number of SEG frames/image slices.

We propose to allow for referencing the series that was used for creating SEG at the series level using SourceSeriesInformationSequence and SourceSeriesSequence (CP 2296 Provide additional ROI parameters to avoid parsing strings).

[fedorov]

@dclunie says we can include references to the segmented series in the shared FG sequence.

[dclunie]

More specifically, the DerivationImageMacro (that includes the references to all the images in the series) can be included at the Shared FG level, rather than including a reference to one image (or all the images) in the Per-Frame FG for each segmented frame - note that the Series per se is still not referenced by this means, since the references are all at the instance level, but the required (satisfied conditional) presence of the Common Instance Reference Module will include the referenced Series information, if that is what is wanted by the consumer.

[fedorov]

@pieper @ReubenDo I finally added the feature that allows to include references to the source images in the cases where segmentation slice orientation does not match that of the source image. All you need to do is add the flag --referencesGeometryCheck 0, which will add all references in the source DICOM directory to the DICOM SEG conversion result.

I tested this on one of the segmentations from the ReMIND collection, and it works as expected (including that MR series is loaded automatically in Slicer when you load SEG).

./bin/itkimage2segimage --inputMetadata ./tumor_seg/meta.json --inputDICOMDirectory ./mr --inputImageList ./tumor_seg/1.nrrd --outputDICOM ./seg1.dcm --referencesGeometryCheck 0

Result:

(0008,1030) LO [Preop]                                  #   6, 1 StudyDescription
(0008,103e) LO [tumor seg - MR ref: 3D_SAG_T1_MP2RAGE]  #  38, 1 SeriesDescription
(0008,1090) LO [https://github.com/qiicr/dcmqi]         #  30, 1 ManufacturerModelName
(0008,1115) SQ (Sequence with undefined length #=1)     # u/l, 1 ReferencedSeriesSequence
  (fffe,e000) na (Item with undefined length #=2)         # u/l, 1 Item
    (0008,114a) SQ (Sequence with undefined length #=176)   # u/l, 1 ReferencedInstanceSequence
      (fffe,e000) na (Item with undefined length #=2)         # u/l, 1 Item
        (0008,1150) UI =MRImageStorage                          #  26, 1 ReferencedSOPClassUID
        (0008,1155) UI [1.3.6.1.4.1.14519.5.2.1.225103893559253813963376864527105591437] #  64, 1 ReferencedSOPInstanceUID
      (fffe,e00d) na (ItemDelimitationItem)                   #   0, 0 ItemDelimitationItem
      (fffe,e000) na (Item with undefined length #=2)         # u/l, 1 Item
        (0008,1150) UI =MRImageStorage                          #  26, 1 ReferencedSOPClassUID
        (0008,1155) UI [1.3.6.1.4.1.14519.5.2.1.90980816060468083041542135051875508914] #  62, 1 ReferencedSOPInstanceUID
      (fffe,e00d) na (ItemDelimitationItem)                   #   0, 0 ItemDelimitationItem
      (fffe,e000) na (Item with undefined length #=2)         # u/l, 1 Item

You will need to use the latest release https://github.com/QIICR/dcmqi/releases/tag/latest or v1.3.4.

I don't know if you are still working on that project or have plans to update the dataset, but in case you do - now you can.