Brain extraction works incorrectly

[d3mage]

The problem The brain_extraction function doesn't extract the brain properly for some images without visible triggers. I was wondering whether there is something wrong with the images, but I don't seem to find any deviations.

import ants
from antspynet.utilities import brain_extraction

def to_hwc(img: np.array) -> np.array:
    if img.shape[2] < img.shape[0] and img.shape[2] < img.shape[1]:
        return img
    if img.shape[1] < img.shape[0] and img.shape[1] < img.shape[2]:
        img = img.transpose((0, 2, 1))
    elif img.shape[0] < img.shape[1] and img.shape[0] < img.shape[2]:
        img = img.transpose((1, 2, 0))
    return img

for patient_folder in patients: 
    annotations = glob(f"{patient_folder}/ann/*")
    for ann in annotations:
        with open(ann, "r") as file:
            modality = json.load(file)["objects"][0]["tags"][0]["name"]
            print(modality)
        volume_path = ann.replace("ann", "volume").replace(".json", "")
        mask_path = glob(f"{volume_path.replace('volume', 'mask')}/*")[0]
        volume_ants = ants.image_read(volume_path)
        mask_ants = ants.image_read(mask_path)

        upd_volume = to_hwc(volume_ants.numpy())
        upd_mask = to_hwc(mask_ants.numpy())
        # print(upd_image.shape)

        volume_ants = ants.from_numpy(
            upd_volume,
            spacing=volume_ants.spacing,
            origin=volume_ants.origin,
            direction=volume_ants.direction,
        )
        mask_ants = ants.from_numpy(
            upd_mask,
            spacing=mask_ants.spacing,
            origin=mask_ants.origin,
            direction=mask_ants.direction,
        )

        extraction_modality = ''

        if 'T1' in modality:
            extraction_modality = 't1'
        elif 'FLAIR' in modality:
            extraction_modality = 't2'
        elif 'T2' in modality: 
            extraction_modality = 't2star'

        # print(ann)
        print(extraction_modality)
        print(volume_ants)
        print(get_image_stats(upd_volume))
        prob_brain_mask = brain_extraction(volume_ants, modality=extraction_modality)
        brain_mask = ants.get_mask(prob_brain_mask)

        explore_3D_array_comparison_mask(volume_ants.numpy(), brain_mask.numpy(), mask_ants.numpy())

        explore_3D_array_with_mask_contour(volume_ants.numpy(), brain_mask.numpy())

    break

A few images to showcase the problem

Most of the images work perfectly fine and the brain is extracted

However, some images result into various artifacts

[cookpa]

I think this is because the image headers say they are in LPI orientation but the anatomical orientation is completely different.

If you load your images into an ITK viewer like ITK-SNAP, the anatomical labels (L, R, A, P, I, S) should be correctly positioned. This doesn't mean that the voxels have to be ordered in a a particular way on disk, just that the header correctly identifies which voxel axis corresponds to which anatomical axis.

[d3mage]

I have looked into .nrrd headers and found that it has RAS orientation, your suggestion was correct. However, upon correcting the orientation with reorient_image2, I get the same mask but in a different direction.

OrderedDict([('type', 'uint16'), ('dimension', 3), ('space', 'right-anterior-superior'), ('sizes', array([241,  18, 384])), ('space directions', array([[ 0.6771    ,  0.        ,  0.        ],
       [ 0.        ,  7.68738457, -0.44059384],
       [ 0.        ,  0.03874365,  0.67599064]])), ('endian', 'little'), ('encoding', 'gzip'), ('space origin', array([ -79.252001  ,  -80.45659621, -140.57083181]))])
RAS
ANTsImage (RAS)
     Pixel Type : float (float32)
     Components : 1
     Dimensions : (241, 384, 18)
     Spacing    : (0.6771, 7.7, 0.6771)
     Origin     : (-83.252, -2864.4703, -297.8264)
     Direction  : [ 1.      0.      0.      0.      0.9984  0.0572  0.      0.0572 -0.9984]

t2star
ANTsImage (RAS)
     Pixel Type : float (float32)
     Components : 1
     Dimensions : (241, 384, 18)
     Spacing    : (0.6771, 7.7, 0.6771)
     Origin     : (-83.252, -2864.4703, -297.8264)
     Direction  : [ 1.      0.      0.      0.      0.9984  0.0572  0.      0.0572 -0.9984]

Mean Intensity: 334.6416931152344
Median Intensity: 138.0
Standard Deviation of Intensity: 468.2676086425781
Min Intensity: 0.0
Max Intensity: 4095.0

[ntustison]

My guess is the orientation is still incorrect. Please verify in ITK-SNAP and/or post the image so we can try to reproduce the problem.

[d3mage]

https://fex.net/s/k2mc2sv

[ntustison]

>>> t = ants.image_read("1.3.6.1.4.1.23438.1.1.0363010001.20160819.43211.5.3.nrrd")
>>> mask = antspynet.brain_extraction(t, modality="t2star")
>>> mask = ants.threshold_image(mask, 0.5, 1, 1, 0)
>>> ants.image_write(mask, "mask.nii.gz")

Okay, so the direction is fine but the anisotropy and partial view are definitely a problem. However, as you can see, I'm not getting the mask that you are. One other possibility is that you need to clear out your cache ("~/.keras/antsxnet) to get any updated weights/data.

[d3mage]

I got my images from a neurosurgeon and he doesn't seem to know much about the MRIs he gives to me. Could you, please, be my source of knowledge a little bit more and explain what's so problematic about the image I sent you? (I would appreciate it if you give me any links etc.)

When I run your snippet, I get the same results However, it doesn't work with my code. I think I will fiddle with it more.

Before I sent this message, I found out that my to_hwc message was causing some ruckus and I got the correct mask.