Convert LIDC-IDRI analysis results to DICOM representation

[fedorov]

Given the dataset analyzed and presented in the supplement to the Cancer Research paper (still under review?), develop scripts and generate a DICOM representation of nodule segmentations as DICOM Segmentation image object, and extracted radiomics features, as a DICOM TID1500 Structured Report. The actual conversion will be done using dcmqi.

[fedorov]

@JoostJM whatever process you (or someone else, I don't know who processed it) used to prepare the reconstructions of the LIDC-IDRI dataset, it should be fixed.

The reconstructed volume is flipped and has origin at 0:

loaded from DICOM	loaded from Reconstructions/image.nii.gz

As an aside, I finally found a use case where volume rendering is really very helpful!

[fedorov]

So it looks like [1] it was flipped and then translated by the X and Y of the original origin, plus shifted by 10 mm in the superior direction. Wow. I wonder if the same "approach" was applied to each case. We should do something so this does not happen for the cases that will be processed in the future.

[1] "Looks like" steps:

1. Geometry information for the image loaded from DICOM:

vs "reconstructed" volume:

2. "Educated" guess for the initial transform + tinkering with the SI translation based on visual appearance of misalignment in volume rendered view in slice thickness increments. Result:

3. Resample the transformed "reconstructed" volume to the matrix of the DICOM-loaded volume using Slicer "Resample Images" module and nearest neighbor interpolator.

4. Calculate the difference image using Slicer "Simple Filters" module. Resulting image is all 0.

[JoostJM]

@fedorov, to the best of my knowledge, @blezek performed the conversion using https://github.com/blezek/LIDCTooling/tree/on_radiomics.

Although the volumes are flipped and the origin is changed, this should not have affected the calculated features, as the spacing did not change and the features are rotation independent.

[fedorov]

@JoostJM thanks for the pointers! I was not worried about the feature calculation being affected, but about data inconsistency that causes problems for sharing and making sure other people can reuse these results in the context of the original DICOM dataset.

@blezek is this the code that does volume reconstruction and writes out the volume: https://github.com/blezek/LIDCTooling/blob/master/src/main/java/com/yittlebits/lidc/Segmenter.java#L309-L340? Is there a particular reason you did volume reconstruction in a way that does not maintain consistency with the DICOM data?

[blezek]

@fedorov, ahh, you are looking behind the curtain! I didn't consider DICOM consistency, more for expediency than anything else. It would be fairly trivial to add orientation information to the NIFTI, or, better, use some of QIICR tools, eg https://github.com/QIICR/dcmqi

[fedorov]

@blezek I ran into this issue exactly in the process of converting data using some of QIICR tools, eg https://github.com/QIICR/dcmqi ;-) While converting segmentations into DICOM, we look for matching slices of the DICOM image series to establish references. This process is done (at this point) by checking whether the origin of the slice is within a slice of the source DICOM series. At the moment, segmentations are "in a different space".

Do you have a quick and reliable way to fix the orientation/origin of the segmentations you prepared such that they match the coordinate system of the source DICOM series? Otherwise I will have to hack this around.

In terms of expediency, might help for the next time to consider this tool for example: https://github.com/icometrix/dicom2nifti.

[JoostJM]

In terms of expediency, might help for the next time to consider this tool for example: https://github.com/icometrix/dicom2nifti.

@fedorov, I'm looking at the tool, but it also uses instance number to sort the dicomfiles: https://github.com/icometrix/dicom2nifti/blob/master/dicom2nifti/convert_generic.py#L41

Still, it does have a function to validate Slice increment, which can be enabled optionally as far as I can see. What do you think?

[pieper]

For reference here's the code in slicer that sorts by image position patient along the vector perpendicular to the slice plane:

https://github.com/Slicer/Slicer/blob/master/Modules/Scripted/DICOMPlugins/DICOMScalarVolumePlugin.py#L240-L300

On Wed, Jul 12, 2017 at 11:26 AM, Joost van Griethuysen < notifications@github.com> wrote:

In terms of expediency, might help for the next time to consider this tool for example: https://github.com/icometrix/dicom2nifti.

@fedorov https://github.com/fedorov, I'm looking at the tool, but it also uses instance number to sort the dicomfiles: https://github.com/icometrix/dicom2nifti/blob/master/ dicom2nifti/convert_generic.py#L41

Still, it does have a function to validate Slice increment https://github.com/icometrix/dicom2nifti/blob/master/dicom2nifti/common.py#L439, which can be enabled optionally https://github.com/icometrix/dicom2nifti/blob/master/dicom2nifti/convert_generic.py#L60 as far as I can see. What do you think?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Radiomics/pyradiomics/issues/269#issuecomment-314805245, or mute the thread https://github.com/notifications/unsubscribe-auth/AAHsfU8ZzPSRwBFWx5k1Ogqt0v-WpIfRks5sNOWrgaJpZM4OQGSR .

[fedorov]

it also uses instance number to sort the dicomfiles

@JoostJM thank you for pointing this out.

I did not look at the code, and just tested with a dataset that had a missing slice. It correctly reported inconsistent spacing (see https://github.com/Radiomics/pyradiomics/issues/263#issuecomment-313464537), but it appears that this was just one of the checks. I want to spend a bit more time a bit later to investigate to get a better idea about robustness of this converter.

[blezek]

In terms of expediency, might help for the next time to consider this tool for example: https://github.com/icometrix/dicom2nifti.

Converting DICOM to NIFTI to DICOM consistently is rather difficult, non-trivial, and I still sometimes get it wrong after doing it for a decade(s). 😃

@fedorov, How can I help out? I converted the volumes in the most simple way possible, only preserving spacing, and not orientation, etc. It was difficult to convert the segmentations from the XML raster representation to DICOM, and took quite a bit of fooling around to get it to work properly.

I could modify the Java code to be more consistent if that would help? Thankfully, the CT are axial and it would (should?) be trivial to extract the DICOM orientation matrix and apply to NIFTI.

[fedorov]

Converting DICOM to NIFTI to DICOM consistently is rather difficult, non-trivial, and I still sometimes get it wrong after doing it for a decade(s).

@blezek sorry for not being clear. I am not saying that DICOM->NIFTI->DICOM is inconsistent. The issue is that the way DICOM->NIFTI conversion was done resulted in a 3D image volume that did not line up with the input DICOM. And so "the terrible hacks" ™ ensued.

I want to spend a bit more time [with dicom2nifti] a bit later to investigate to get a better idea about robustness of this converter.

Indeed, it is not ready to use, unfortunately. I reported two issues that came in the process of testing it on some sample datasets, see https://github.com/icometrix/dicom2nifti/issues/6 and https://github.com/icometrix/dicom2nifti/issues/7.

[fedorov]

As discussed with @pieper earlier, https://github.com/rordenlab/dcm2niix might be a more robust alternative for DICOM conversion than dicom2nifti. I will add notes here if I encounter issues after testing.

[JoostJM]

@fedorov, is it an idea to share a link to the dataset here?

[fedorov]

I never finished the conversion, thus the issue is still open...

Do you mean to add the Dropbox folder link that with the segmentation dataset you prepared that we shared for the evaluation?

[JoostJM]

Do you mean to add the Dropbox folder link that with the segmentation dataset you prepared that we shared for the evaluation?

Yes, as this is already converted with segmentations. it's not the full set, but a sizeable start.

[JoostJM]

Alternatively, we could also share this set on the radiomics.io page. @ahmedhosny, @hugoaerts, what do you think of this?

[ahmedhosny]

can add it no problem - I have a few other outstanding updates, so I'll do them all together.

[JoostJM]

@ahmedhosny, do you have a status update on this issue?

[fedorov]

@JoostJM we could also contribute normalized labels back to TCIA. Do you think we should have segmentations of all nodules for completeness, or those that you prepared are sufficient? I am personally not sure what is the benefit of sharing the segmentations on radiomics.io. I am also not sure if you suggested hosting of those on radiomics.io, or just adding a pointer to the Dropbox folder?

[JoostJM]

@fedorov, I'm fine with both sharing the data directly on radiomics.io or just sharing a pointer to the dropbox folder. My main idea here was to make it publicly available also to people we don't directly know.

As to sharing the results back to TCIA, We can surely share the dataset I prepared, but as to converting the other segmentations, that depends a bit on how much work that would be. I don't have the conversion code to deal with the xml-segmentation files, I converted niftii's @blezek converted, with manual checks for correctness.

[ahmedhosny]

Great, just send me a link - would those go under "data" or "code"?

[JoostJM]

I'd say "data",@hugoaerts?

@fedorov, I created a share link for the repository, i'll email it to you and @ahmedhosny.

[blezek]

@JoostJM, Can you send the link to me as well? I have a colleague who is interested in the corrected data.

[jvanlunenburg]

I was wondering did you guys publish the converted data and extracted radiomics anywhere? Specifically I'd be interested to know the (302) patient IDs used from the LIDC cohort :)

[fedorov]

https://aapm.onlinelibrary.wiley.com/doi/full/10.1002/mp.14445