Support for forcing 3D NIFTI output (no intelligent merging) for vNav conversion

[tokeefe]

When converting a vNav series to BIDS, dcm2niix is converting every instance as a separate file. This is all documented in Issue #208. The problem I'm seeing is that dcm2niix is creating a mixture of 3D and 4D files in some cases. In my case, I have 144 x DICOM instances, but after conversion I end up with 142 NIFTI+JSON files. It seems that 2 pairs of DICOM instances are being merged together into 4D NIFTI files and the rest are emitted as 3D files. This is problematic because the JSON sidecar for the 4D files only contains the "ImageComments" (which is where the vNav motion estimates are embedded) from the 1st instance. I could bypass this issue if dcm2niix contained an argument e.g., "Output a single 3D NIFTI+JSON for every instance encountered. Do not merge anything into 4D." which might also be useful for splitting multi-echo BOLDs and possibly other use cases.

[neurolabusc]

@tokeefe can you send me a DropBox/GoogleDrive/etc link of a DICOM series that exhibits this issue (my email is in my avatar). As noted in issue 208 the varying spatial transforms in a vNav dataset is incompatible with the 4D NIfTI format (which requires a single spatial transform for all volumes). I am not sure if there is a universal method for detecting vNavs. However, I think we agree that the expected behavior would be to save every volume in a vNav series as a separate 3D volume (e.g. 144 volumes in your case).

I have only seen vNav data for Siemens V-series DICOM (e.g. Prisma VE11C) not X-series enhanced DICOM (e.g. Vida XA11). If any X-series users have vNav data, I would be interested in seeing an exemplar.

[neurolabusc]

I have looked at this a little bit. I assume you are using the MGH vNav WIP, not a Siemens product sequence. As I understand it, the first two volumes of the vNav series are perfectly aligned with each other, and can be saved as a single 4D NIfTI file, while subsequent volumes have different spatial transforms so must be saved as separate 3D NIfTI files. I would assume that most users discard the first two volumes for T1-effects. The current solution of dcm2niix seems valid, and changing the behavior may have unintended consequences for users like @mharms who may prefer volumes to be combined when possible. Finally, I am always reluctant to make bespoke changes for work-in-progress sequences, as this increases the complexity of dcm2niix and can be impacted by changes implemented in these sequences. My sense is that if you want to change the behavior of dcm2niix, you should make a fork of dcm2niix to handle this situation. The modification is very easy if you assume your data will always be stored as mosaics, but more involved if each 2D slice is saved as a separate DICOM. Having a robust method to detect the vNav would help. I do not think there are any public tags for this, though I think you can detect this sequence using a (computationally costly) dive into the CSA headers.

Therefore, I would propose this issue gets a "wontfix" label.

[mharms]

FWIW, I believe that we are still using a "2017" version for conversion of our HCP-Lifespan acquisitions, and as I noted in #208, we locally handle the merging into a 4D volume, with arrayed accompanying json, via custom local code. Sounds like we would probably need to modify our local code to work with the current version of dcm2niix. Since our conversions are working fine for our purposes, we have had no reason to do so.

[tokeefe]

I have looked at this a little bit. I assume you are using the MGH vNav WIP, not a Siemens product sequence.

Thanks for taking the time to look into this. Yep, this is a MGH vNav.

As I understand it, the first two volumes of the vNav series are perfectly aligned with each other, and can be saved as a single 4D NIfTI file, while subsequent volumes have different spatial transforms so must be saved as separate 3D NIfTI files. I would assume that most users discard the first two volumes for T1-effects. The current solution of dcm2niix seems valid, and changing the behavior may have unintended consequences for users like @mharms who may prefer volumes to be combined when possible.

Yes, in the vNav scan I'm looking at (working on getting approval to share, though maybe this isn't needed anymore?) the 1st and 2nd volumes (AcquisitionNumber 1 and 2) are being merged together. Those volumes indeed have identical orientations (0020, 0037) and should probably be tossed anyway. However, there are also two additional volumes (AcquisitionNumber 52 and 71) that are being merged together. The orientations between those volumes do not appear identical and the ImageComments are also different. Only the ImageComments from AcquisitionNumber 52 end up in the JSON sidecar.

Finally, I am always reluctant to make bespoke changes for work-in-progress sequences, as this increases the complexity of dcm2niix and can be impacted by changes implemented in these sequences. My sense is that if you want to change the behavior of dcm2niix, you should make a fork of dcm2niix to handle this situation. The modification is very easy if you assume your data will always be stored as mosaics, but more involved if each 2D slice is saved as a separate DICOM.

Understandable. These vNavs indeed appear to be a WIP and possibly a moving target. I was trying to take this into consideration by proposing a more generic command line argument to have dcm2niix emit 3D volumes at most, regardless of scan type. Of course my motivation was to support these vNavs, so it's very likely this feature wouldn't be very useful outside that context. Also, I could always scan the DICOMs externally and invoke dcm2niix -s y on each one individually. This would be doable now without needing any changes to the core tool (which is superb btw).

Having a robust method to detect the vNav would help. I do not think there are any public tags for this, though I think you can detect this sequence using a (computationally costly) dive into the CSA headers. Therefore, I would propose this issue gets a "wontfix" label.

If I come across a way to robustly detect vNavs, I'll certainly toss it your way!

[neurolabusc]

Hmm, I have only seen exemplars where the first pair were identical. Without seeing an example, it would be hard to create a good solution. Feel free to make a fork and develop your own solutions, you can share your fixes as a pull request if you wish.

The undocumented option -z 3 may be what you are looking for (e.g. dcm2niix -z 3 /path/to/DICOMs). This will save each volume as an uncompressed 3D file.

[neurolabusc]

Please test this simple solution. It assumes vNav data is always saved as mosaics. Hopefully, this does not have unintended consequences for other datasets.

git clone --branch development https://github.com/rordenlab/dcm2niix.git
cd dcm2niix/console
g++  -I.  main_console.cpp nii_foreign.cpp nii_dicom.cpp jpg_0XC3.cpp ujpeg.cpp nifti1_io_core.cpp nii_ortho.cpp nii_dicom_batch.cpp  -o dcm2niix -DmyDisableOpenJPEG
./dcm2niix ....

[tokeefe]

Thank you so much for putting time into this. Using the vNav I shared with you, this development version is now producing 3D files for all instances including AcquisitionNumber 1 and 2 (InstanceNumber 1 and 33) but for some reason it's still merging AcquisitionNumber 52 and 71 (InstanceNumber 1633 and 2241). I've started to poke around the source and I'm sort of hovering around this section here but I haven't drawn any conclusions yet.

[neurolabusc]

Can you make sure you are using the latest commit on the development branch (v1.0.20210222)? When I run this, all instances are stored as separate 3D files.

[tokeefe]

No problem. This is exactly what I'm doing, note for note

Cone repo

$ git clone --branch development https://github.com/rordenlab/dcm2niix.git
$ cd dcm2niix/console

Double check version

$ cat nii_dicom.h | grep '#define kDCMdate'
#define kDCMdate "v1.0.20210222"

$ git rev-parse HEAD
ae16a61c81e158c9207501fdc12c322709ffe40b

Compile

$ g++  -I.  main_console.cpp nii_foreign.cpp nii_dicom.cpp jpg_0XC3.cpp ujpeg.cpp nifti1_io_core.cpp nii_ortho.cpp nii_dicom_batch.cpp  -o dcm2niix -DmyDisableOpenJPEG

Extract DICOMs

$ mkdir dicoms
$ tar xpfvz /tmp/vnav.tar.gz -C dicoms

Run dcm2niix

$ mkdir bids
$ ./dcm2niix -b y -z y -o ./bids ./dicoms > output.txt

Check number of occurrences of 'Convert 2' in stdout

$ cat output.txt | grep 'Convert 2'
Convert 2 DICOM as ./bids/<trim>_i01633 (32x32x32x2)

No file for instance *i02411.json

$ ls bids/*i02241.json

[neurolabusc]

@tokeefe I think the latest commit to the development branch (v1.0.20210228) provides a robust solution to this problem. Please test and close if this resolves this issue.

[tokeefe]

That definitely seems to have done the trick. Thank you!