search

moloney / dcmstack

DICOM to Nifti conversion with meta data preservation
Other
72 stars 51 forks source link

Problems with large multisequence data #11

Closed chrisgorgo closed 11 years ago

chrisgorgo commented 11 years ago

Hi, I got a problematic dataset to convert coming from our 7T scanner. Not only it is big (5GB uncompressed) but also includes multiple sequences (and at least one of them is multivolume). There is no way to separate the sequences using filenames, but even if I try to sort them by sequence name taken from the dicom headers dcmstack fails (and it's not because it runs out of memory). I've been told that 7T is the future so it would be cool if dcmstack would support such datsets ;)

https://www.dropbox.com/s/s731ex9oaghx9i0/Vaso.tar.xz

chrisgorgo commented 11 years ago

And here are the niftis converted using "isisconv -in /tmp/Vaso -out /tmp/S{sequenceNumber}_{sequenceDescription}_e{echoTime}.nii"

https://www.dropbox.com/s/5h3vqzs45kyhsl1/niftis.tar.gz

DerOrfa commented 11 years ago

Just to note, thats the "isisconv" from there.

moloney commented 11 years ago

Hello,

Thanks for the feedback!

The only series I was seeing errors with was the "pilot" series since it contains non-parallel slices. I've known about this issue for a while, but wasn't very motivated to fix it since it is rare that people need to convert these acquisitions. I went ahead and fixed this issue with commit 85d7d11de54631e031924566f11957937c8a669f.

It is highly recommended that the data is sorted into one series per directory before conversion to reduce memory usage. However, while testing with your data I noticed the memory usage (which is admittedly already not great) was continuing to go up from one directory to another. This issue should be fixed with commit c9f858b25abbe0e4b425f8648ae26fbfad6c047f.

While testing with your data I was also inspired to try to reduce memory consumption and processing time when the meta data is not being embedded/dumped. I have done some quick testing where I define a minimal meta extractor that only pulls out the DICOM elements required for conversion to Nifti. My initial results are promising with the memory usage reduced by a factor of three and the run time reduced by a factor of six (so the run time is basically competitive with dcm2nii). I want to do some more testing and cleanup before committing the code, but will post an update here when I do.

Also, thanks for the link to the ISIS software. Looks interesting.

moloney commented 11 years ago

Ok, the improvements to memory usage and speed are in commit c2fd8b0ff3f022387dc06cdb1306ab32783678d6.

Please try it out.