search

Deep-MI / FastSurfer

PyTorch implementation of FastSurferCNN
Apache License 2.0
460 stars 120 forks source link

Holes in brain mask #44

Closed danielFangstrom closed 1 year ago

danielFangstrom commented 3 years ago

Hi all! First, let me just say that FastSurfer is a great tool and I am impressed with what it can do. That being said, I do have some issues with the masks created in the process. The masks consistently have holes in them, ranging from one single voxel to several as in the provided screenshot.

brain_mask_holes

This particular mask was created by running the script without any extra parameters (so just specifying the input image and the output directory and subject id). The input image is a UNIDEN image (T1w denoised) with the dimensions 192x240x256 (which gets converted to 256x256x256 by FS). It’s entirely possible it is all because of me doing something strange so any help is appreciated!

All the best, Daniel

LeHenschel commented 3 years ago

Hey Daniel,

Thanks for your message. I have not seen this error in any of the images I processed so far. Would it be possible for you to share one of the input images with us? And does the original segmentation (aparc+aseg.orig.mgz) also contain holes?

Best, Leonie

m-reuter commented 3 years ago

Hi Daniel. Don't use de-noised images. There could be easy too much niceifying (they often look like perfect paintings and can remove a lot of valuable information. Try the original scan. Best Martin

danielFangstrom commented 3 years ago

Hi Leonie and Martin, Thanks for your quick replies, and thanks for pointing out the issue with denoised images! I have now tried it with a non-denoised image, and while it didn't produce any holes in the mask, the skull stripping is not working properly. As you can see, there are many places where the skull is still present.

skullstrip_issues

This is the T1 image I used as input. (Just a screenshot but let me know in case you'd want the header info/mri_info output. Sharing the actual image might be somewhat tricky.) I also added the recon-all.log-file, in case you would be able to find anything useful in there: recon-all.log

T1_input

Are there any changes you can already recommend that might help with the incomplete skull stripping?

Thanks in advance!

UNIDEN

For the sake of completeness, I added the denoised UNIDEN image I used as the input initially.

UNIDEN_input

The aparc+aseg.orig in that case has no obvious holes that I can see.

aparc_aseg_orig_UNIDEN

m-reuter commented 3 years ago

Hi Daniel, Oh, I see. Sorry, but our method does not support 7T scans yet. It is trained and tested only for T1 weighted images at 3T (for example it should work very well for MPRAGE or multi-echo MPRAGE from 3T scanners).

danielFangstrom commented 3 years ago

Hi Martin, These scans were actually acquired using 3T, but the sequence was adopted from one used at 7T. I can however imagine that causes issues still. Let me know in case you think there are any things I could do to make this sequence work in FastSurfer. Regardless, thanks for your help and your input!

All the best, Daniel

LeHenschel commented 3 years ago

Hey Daniel,

The UNIDEN image looks very dark towards the center. Could you display the the aparc+aseg.orig with a colormap? I think it is possible, that the network labels voxels in the middle as background which leads to the holes in the mask (this can not really be seen in the binary map screenshot because the center is completely black).

For the skull strip issue, does the mask (and the aparc+aseg.orig) also include the voxels which are left afterwards?

Best, Leonie

m-reuter commented 3 years ago

Both these images (normalized and unnormalized) have a very different intensity profile from anything the network has ever seen in training. Which is why segmentation fails and as a consequence brain mask. The only way to extend the network is to train with these cases (if freesurfer provides any meaningful labels on these). Another alternative could be to synthesize these images from mprage and do augmentation in training. Both options are not anything we could do easily.

danielFangstrom commented 3 years ago

That's too bad but does indeed make sense with how the output looked in both cases. I'll try to find some alternative ways to process these images. Thank you both for your help and for your time!

All the best, Daniel

runjia0124 commented 1 year ago

I have the same issue running on a HCP Young Adult scan, Subject id 102614, I used an ANTs-structurally corrected T1w scan and found a big hole on my brain mask.

m-reuter commented 1 year ago

What version did you run?

LeHenschel commented 1 year ago

On the original HCP scan this works fine. Maybe try running it on that instead of the ANTs-Structurally corrected one.

runjia0124 commented 1 year ago

Hi Leonie,

Thank you for the reply and apologies for my delay. The hole appears on a HCP Young Adult scan, Subject id 103010, not 102614, sorry for the mistake. I ran the current Fast-surfer stable version, cloned this repo directly and ran it on my local machine. I tried the original T1w without structural correction but still found a hole in the mask, segmentation works well. For you to reproduce the bug, this is the specific T1w scan I ran with: 103010/unprocessed/3T/T1w_MPR1/103010_3T_T1w_MPR1.nii.gz. Please note that this is the unpreprocessed release of the HCP Young Adult. Please let me know if more information is needed.

Best, Runjia