Segmentation Inaccuracies with degenerated hippocampi

[mchoi99]

Hi there,

I am currently trying to segment the hippocampal subfield of AD and MCI patients using hippunfold, and it seems like there are many inaccuracies with the segmentation outputs (ie. segmentations crossing into CSF space, the hippocampus not looking like a hippocampus, etc.) and I am wondering if there is any additional code I should be using to correct these errors.

Many times, even cognitively normal subjects seem to output errors in segmentation, particularly those who longitudinally acquire MCI or AD. Perhaps any changes to the overall cortical surface is causing errors in segmenting the hippocampus.

I would really appreciate some insight from the HippUnfold team on this issue. Thank you.

[jordandekraker]

Happy to help troubleshoot, but note that these models weren't trained with much disease data and so segmentation may not work super well if there is very high atrophy. Still, its worth taking a look at some failed cases to try and see why. Could you 1) check the "qc" output folder and 2) post a few screenshots of cases that failed?

[mchoi99]

Hi Dr. Jordan Dekraker,

Thank you for your quick response. I am attaching a screenshot of the space-cropT2w_desc-subfields_atlas-multihist7_dseg.png from the qc folder, as well as my own screenshots from ITK-SNAP.

This image is of a subject who developed MCI, and these images are baseline images when they were cognitively normal.

Thank you once again.

[image: sub-4179_hemi-R_space-cropT2w_desc-subfields_atlas-multihist7_dseg.png] [image: Screenshot from 2024-03-18 16-59-47.png][image: Screenshot from 2024-03-18 17-00-09.png]

On Mon, Mar 18, 2024 at 4:06 PM Jordan DeKraker @.***> wrote:

Happy to help troubleshoot, but note that these models weren't trained with much disease data and so segmentation may not work super well if there is very high atrophy. Still, its worth taking a look at some failed cases to try and see why. Could you 1) check the "qc" output folder and 2) post a few screenshots of cases that failed?

— Reply to this email directly, view it on GitHub https://github.com/khanlab/hippunfold/issues/287#issuecomment-2005129472, or unsubscribe https://github.com/notifications/unsubscribe-auth/BHCVWZJB4T6QJ7UWSUVCQCLYY5QPPAVCNFSM6AAAAABE4MTRQ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBVGEZDSNBXGI . You are receiving this because you authored the thread.Message ID: @.***>

[jordandekraker]

I don't think your images got attached properly, do you think you could try again please? Make sure to add them on the Git Issue and not by email

Thanks

[mchoi99]

Hi Dr. Jordan Dekraker,

My apologies, here are the screenshots. I am attaching a screenshot of the space-cropT2w_desc-subfields_atlas-multihist7_dseg.png from the qc folder, as well as my own screenshots from ITK-SNAP.

[jordandekraker]

Thanks This definitely looks like a gross oversegmentation, particularly in th eposterior areas. This is a failure at the UNet tissue segmentation step, which we noted occurs a bit more often in T2w images. We do have a few other options of trained UNets that could help, I would suggest: --force-nnunet-model synthseg_v0.2 That should provide equally good (or better) results, with hopefully no more gross errors. If there are still gross errors, then it might be worthwhile to instead run on a more standard T1w image. We found this to be more reliable, but sometimes shows less subject-specific detail.

Let me know how that works

[mchoi99]

Hi there,

Thank you for the quick response! So should I just include that bit of code in the tail of the original code? The code would look something like this: "sudo docker run -it --rm -v /home/peter/Desktop/CNcombined/CN1:/bids -v /media/peter/DATA/CNhippunfoldT1T2/CN1_hippunfold:/output khanlab/hippunfold:latest /bids /output participant --modality T2w --t1-reg-template -p --cores 20 --force-nnunet-model synthseg_v0.2"

Please let me know if there is anything obvious I am missing here.

I will also try to run the code with just T1w images as well.

Thank you again. Peter

On Tue, Mar 19, 2024 at 10:54 AM Jordan DeKraker @.***> wrote:

Thanks This definitely looks like a gross oversegmentation, particularly in th eposterior areas. This is a failure at the UNet tissue segmentation step, which we noted occurs a bit more often in T2w images. We do have a few other options of trained UNets that could help, I would suggest: --force-nnunet-model synthseg_v0.2 That should provide equally good (or better) results, with hopefully no more gross errors. If there are still gross errors, then it might be worthwhile to instead run on a more standard T1w image. We found this to be more reliable, but sometimes shows less subject-specific detail.

Let me know how that works

— Reply to this email directly, view it on GitHub https://github.com/khanlab/hippunfold/issues/287#issuecomment-2007679335, or unsubscribe https://github.com/notifications/unsubscribe-auth/BHCVWZJ5B5MJFM4B4PU4JKDYZBUTNAVCNFSM6AAAAABE4MTRQ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBXGY3TSMZTGU . You are receiving this because you authored the thread.Message ID: @.***>

[jordandekraker]

yup, that will work! I recommend writing to a new output folder though if that is the same output name you used previously

[akhanf]

Hi Peter, you're working with Trevor Steve right? Was just mentioning on an e-mail with him that we have some ongoing work and QC on the ADNI data, and will be training a new model to improve performance on elderly/atrophic datasets. Will connect over e-mail on that soon.

My bet is that the synthseg models won't fare much better on these cases, since this kind of level of atrophy isn't seen in that training set either, but worth a shot..

[mchoi99]

Hi Dr. Ali Khan,

Yes! I am a Master’s Student working with Dr. Trevor Steve.

That sounds great and promising - I hope the new model is able to segment atrophied brain much better.

Would you happen to know the estimated timeline for retraining the model?

Thank you, Peter

On Tue, Mar 19, 2024 at 11:23 AM Ali Khan @.***> wrote:

Hi Peter, you're working with Trevor Steve right? Was just mentioning on an e-mail with him that we have some ongoing work and QC on the ADNI data, and will be training a new model to improve performance on elderly/atrophic datasets. Will connect over e-mail on that soon.

My bet is that the synthseg models won't fare much better on these cases, since this kind of level of atrophy isn't seen in that training set either, but worth a shot..

— Reply to this email directly, view it on GitHub https://github.com/khanlab/hippunfold/issues/287#issuecomment-2007742031, or unsubscribe https://github.com/notifications/unsubscribe-auth/BHCVWZKMMLW26V7WIEUJYP3YZBYABAVCNFSM6AAAAABE4MTRQ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBXG42DEMBTGE . You are receiving this because you authored the thread.Message ID: @.***>

[akhanf]

Might be at least a couple weeks @Bradley-Karat is away at a conference this week, and I'm off next week. But will let you know when we have something you can test.

[mchoi99]

Thank you for the update @akhanf.

@jordandekraker I have ran the new code and it seems like there are still over-estimations, this time not in the posterior but more frequent across the body. I have attached some screenshots below for you to review. In particular, the subiculum (red) is segmented as much longer and the CA3 (yellow) is now segmenting in to the CSF at times. These are quite common across the 19 CN subjects I scanned, "highreshippo" T2w images (with T1w) from ADNI.

At this point, would you recommend using the original command line? Or is there a different model that you think could help?Thank you.

[jordandekraker]

Thanks for sharing. Performance definitely looks better now using this model, but I agree there are stil quite a few errors. my suggestions would be: 1) manually check the results and discard subjects with major errors 2) wait for Ali & Brad to share their atrophy-trained model 3) try running with ONLY the T1w image (ie. --modality T1w). This can help if the contrast is low in T2w images, but there's certainly no guaruntee that it will work better since its also not trained with such extensively atrophied cases

I think that's all I can offer for help right now, and I'm definitely looking forward to seeing how Ali&Brad's new model performs too!

[mchoi99]

Thank you so much for your help!

On Thu, Mar 21, 2024 at 11:52 AM Jordan DeKraker @.***> wrote:

Thanks for sharing. Performance definitely looks better now using this model, but I agree there are stil quite a few errors. my suggestions would be:

1. manually check the results and discard subjects with major errors
2. wait for Ali & Brad to share their atrophy-trained model
3. try running with ONLY the T1w image (ie. --modality T1w). This can help if the contrast is low in T2w images, but there's certainly no guaruntee that it will work better since its also not trained with such extensively atrophied cases

I think that's all I can offer for help right now, and I'm definitely looking forward to seeing how Ali&Brad's new model performs too!

— Reply to this email directly, view it on GitHub https://github.com/khanlab/hippunfold/issues/287#issuecomment-2013173551, or unsubscribe https://github.com/notifications/unsubscribe-auth/BHCVWZILJTFEPT4RAAVOOADYZMM7FAVCNFSM6AAAAABE4MTRQ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMJTGE3TGNJVGE . You are receiving this because you authored the thread.Message ID: @.***>

[mchoi99]

Hi there Dr. Dekraker and Dr. Khan,

I hope this message finds you well.

I, along with other labmates, have found a good proportion of CN subjects from ADNI that result in oversegmentations in T2. We would like to ask - are there any necessary pre-processing steps that must be done upon downloading files from ADNI? Does hippunfold have any preprocessing steps integrated into it? What are the main factors that cause major segmentation quality differences between T1w and T2w?

Thank you for your time. Peter

[jordandekraker]

As suggested above, try the T1w images. The T2w images in the ADNI dataset are anisotropic - they have good resolution in a coronal plane but very thick slices and so its hard to get a detailed 3D context. We also noted in the HippUnfold paper that performance was generally more reliable in T1w images, possibly because the contrast between grey and white matter is more optimal. This is difficult to recover in the T2w images, even with careful preprocessing.

[mchoi99]

Hi Dr. DeKraker,

Thank you for your explanation, I will try T1w images. Would you happen to have a potential timeline for retraining HippUnfold?

Thank you! Peter

On Wed, Apr 17, 2024 at 1:50 PM Jordan DeKraker @.***> wrote:

As suggested above, try the T1w images. The T2w images in the ADNI dataset are anisotropic - they have good resolution in a coronal plane but very thick slices and so its hard to get a detailed 3D context. We also noted in the HippUnfold paper that performance was generally more reliable in T1w images, possibly because the contrast between grey and white matter is more optimal. This is difficult to recover in the T2w images, even with careful preprocessing.

— Reply to this email directly, view it on GitHub https://github.com/khanlab/hippunfold/issues/287#issuecomment-2062098245, or unsubscribe https://github.com/notifications/unsubscribe-auth/BHCVWZLAXSRMOVZ6RWQT4DDY53G6RAVCNFSM6AAAAABE4MTRQ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANRSGA4TQMRUGU . You are receiving this because you authored the thread.Message ID: @.***>

[Bradley-Karat]

Hi Peter,

The T1w ADNI specific model has been re-trained, and I am just running some test cases to see how it performs. Will keep this thread updated once thats finished!

[mchoi99]

Hi Dr. DeKraker,

Thank you for the update. How many samples do you need? Do you need both T1w and T2w scans?

Thank you.

On Fri, Apr 19, 2024 at 11:57 AM Jordan DeKraker @.***> wrote:

Well training only takes a couple of days, plus maybe a couple more if we want to augment the training data using SynthSeg as we have been doing in the latest models. However, we'd need access to training data either 1) through manual segmentation or 2) through ADNI cases that were successful (though these are likely to be the easiest cases). If you're willing to share your results then we could apply 2) but otherwise I don't know of anyone who is actively working on this.

Let me know if that's something you're interested in doing!

— Reply to this email directly, view it on GitHub https://github.com/khanlab/hippunfold/issues/287#issuecomment-2067045323, or unsubscribe https://github.com/notifications/unsubscribe-auth/BHCVWZMRASCCTAC6SZ66LFDY6FLIFAVCNFSM6AAAAABE4MTRQ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANRXGA2DKMZSGM . You are receiving this because you authored the thread.Message ID: @.***>

[akhanf]

Hi Peter, as Brad and I mentioned above, we're already training (actually already done, just need to deploy to test) a model with ADNI data. Should have some results to discuss soon, but might be a good idea for us to set-up a call to chat about next steps. Jordan fyi this is an ongoing collaboration we have going with Trevor Steve's lab, so work is already underway here that Brad is helping with.

[mchoi99]

Hi Dr. Ali Khan,

Thank you so much for updating us. Happy to hear that training is completed!

What time next week would you be available for a call?

Peter

On Fri, Apr 19, 2024 at 1:26 PM Ali Khan @.***> wrote:

Hi Peter, as Brad and I mentioned above, we're already training (actually already done, just need to deploy to test) a model with ADNI data. Should have some results to discuss soon, but might be a good idea for us to set-up a call to chat about next steps. Jordan fyi this is an ongoing collaboration we have going with Trevor Steve's lab, so work is already underway here that Brad is helping with.

— Reply to this email directly, view it on GitHub https://github.com/khanlab/hippunfold/issues/287#issuecomment-2067165542, or unsubscribe https://github.com/notifications/unsubscribe-auth/BHCVWZNHEFWFYGNRKVYJSTDY6FVWJAVCNFSM6AAAAABE4MTRQ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANRXGE3DKNJUGI . You are receiving this because you authored the thread.Message ID: @.***>

[mchoi99]

Good morning Dr. Khan, Dr. DeKraker, and Dr. Karat,

I would like to follow up on our previous conversation to get an update on where we are on the retraining and retesting of the new model for the ADNI data set. Our team is eagerly waiting for the new version, and we would love to set up a call whenever you are available this week or the following week.

Thank you again for all your work. Peter

On Fri, Apr 19, 2024 at 1:32 PM Mirsol Choi @.***> wrote:

Hi Dr. Ali Khan,

Thank you so much for updating us. Happy to hear that training is completed!

What time next week would you be available for a call?

Peter

On Fri, Apr 19, 2024 at 1:26 PM Ali Khan @.***> wrote:

Hi Peter, as Brad and I mentioned above, we're already training (actually already done, just need to deploy to test) a model with ADNI data. Should have some results to discuss soon, but might be a good idea for us to set-up a call to chat about next steps. Jordan fyi this is an ongoing collaboration we have going with Trevor Steve's lab, so work is already underway here that Brad is helping with.

— Reply to this email directly, view it on GitHub https://github.com/khanlab/hippunfold/issues/287#issuecomment-2067165542, or unsubscribe https://github.com/notifications/unsubscribe-auth/BHCVWZNHEFWFYGNRKVYJSTDY6FVWJAVCNFSM6AAAAABE4MTRQ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANRXGE3DKNJUGI . You are receiving this because you authored the thread.Message ID: @.***>