search

wasserth / TotalSegmentator

Tool for robust segmentation of >100 important anatomical structures in CT and MR images
Apache License 2.0
1.55k stars 256 forks source link

Output Orientation Issue #20

Closed denbonte closed 2 years ago

denbonte commented 2 years ago

Dear TotalSegmentator devs,

Good day, and thank you so much for the time you put in the development of this amazing toolkit!
I tried TotalSegmentator in a number of cases/using a number o publicly available datasets (and preprocessing tools, e.g., to get from the DICOM images to the input NIfTI files). I have noticed that the orientation of the pixel data in the output NIfTI files doesn't always line up with the one from the input data. In other words - if I start from a DICOM series, convert it to NIfTI using, say, Plastimatch or dcm2niix, process it with TotalSegmentator (using the CLI) and then try to use them for some downstream tasks with any analysis library based on SITK - this will yield a wrong result or raise an error (if the library carries out the proper checks). One example is radiomics features extraction using PyRadiomics (which I know you folks use as well as part of the pipeline, if specified).

As expected, this is what happens when ones tries to visualise the output from the TotalSegmentator pipeline loading the NIfTI files with SITK (for the sake of simplicity, I'm visualising only a few structures here):

image

If the same files are loaded in 3DSlicer - or using nibabel, for instance - everything looks perfect (as both of them will probably apply the rotation encoded in the affine matrix in the loading process):

image

CC'ing @fedorov and @LennyN95 - which are both interested in this. I hope this can somehow save someone hours of debug!

fedorov commented 2 years ago

I hope this can somehow save someone hours of debug!

Not just hours of debug, but potentially a huge embarrassment or disappointment. For someone working with 1000s of CT scans, visual examination of proper alignment between segmentation results and input image volume is just not feasible. It is not impossible for someone to not do visual checks and extract radiomics features, or do visual checks, but have inconsistencies in the cohort where orientation may vary just for some of the cases. This potentially can be a huge issue for radiomics studies.

fedorov commented 2 years ago

@denbonte would it help you think if you could provide a notebook that demonstrates the issue start to finish for a public IDC sample?

denbonte commented 2 years ago

would it help you think if you could provide a notebook that demonstrates the issue start to finish for a public IDC sample?

Definitely, yes. I'm working on it right now and I will hopefully be able to share this later today (or tomorrow!) - so that the devs can see what I mean precisely!

denbonte commented 2 years ago

Sorry all for the multiple posts here!

While preparing the MWE notebook, running everything again in a Colab instance with the latest version of TotalSegmentator, I noticed I cannot reproduce the problem I had last week anymore - using the exact same piece of data from a public collection hosted at IDC.

@wasserth is it possible you folks fixed the bug in the meantime? It would probably have something to do with the way you load the input data and/or save the output data (and potentially some operations you guys applied between the two - although I know TotalSegmentator is based on nnU-Net - and I've never had similar issues with the models shared by Fabian!)

Thanks, Dennis.

wasserth commented 2 years ago

We had issues with image segmentation not properly aligning. I did some fixes in version 1.3 (see https://github.com/wasserth/TotalSegmentator/blob/master/CHANGELOG.md). If you upgraded the version this might have fixed the problem.

denbonte commented 2 years ago

Hey @wasserth!

I did some fixes in version 1.3 (see https://github.com/wasserth/TotalSegmentator/blob/master/CHANGELOG.md).

I think that did the job 🙃

Thank you for all the work - I'm closing the issue!