IntensityRangeStandardization produces negative intensity values

[lazypoet]

I was using MICCAI BRATS 2015 training dataset. I took a list of images and their respective masks.

# f = list of filenames for t2 sequence
im_ar = [sitk.GetArrayFromImage(sitk.ReadImage(i) for i in f]
im_ar = np.array(im_ar)
im_msk = im_ar>0
n = IntensityRandeStandardization()
# train and transform
r, out = n.train_transform([i[m] for i, m in zip(im_ar, im_msk)])
for i, m, o  in zip(im_ar, im_mask, out):
    i[m] = o

When i checked for some generic output o:

[array([-276.6272986 , -264.94980174, -266.89605121, ..., -266.89605121,
        -272.73479964, -272.73479964]),
 [-256.55019873, -145.10180764,  -40.20920427, ...,   18.69724008,
         -53.32077969, -276.21756186],
 array([-242.07590339, -180.49569523, -160.90199263, ..., -256.07140525,
        -261.66960599, -278.46420821]),
 array([-263.61638514, -270.51243865, -261.89237176, ..., -138.62541528,
        -142.93544873, -268.78842527]),
 array([-289.90758914, -190.99232162, -271.58994701, ..., -125.04880994,
          -2.28223993, -125.04880994]),
 array([-271.58994701, -244.11348381, -264.26289016, ..., -262.43112594,
        -247.77701223, -267.92641858]),
 array([-255.26526434, -237.32862916, -247.29342648, ..., -251.27934541,
        -217.39903452, -257.2582238 ]),
 array([-266.97390119, -261.99150253, -266.97390119, ..., -247.04430655,
        -197.22031995, -222.13231325]),
 array([-267.50772962, -262.16944534, -272.8460139 , ..., -239.03688013,
        -247.9340206 , -265.72830153]),
 array([-260.52822525, -241.17910424, -233.92318386, ..., -262.94686538,
        -279.87734627, -282.29598639]),
 array([-268.32900412, -265.59142244, -264.2226316 , ...,   22.02947516,
        -257.3786774 , -265.59142244]),
 array([-278.91700386, -267.92641858, -267.92641858, ..., -289.90758914,
        -286.24406072, -286.24406072]),
 array([-273.54950873, -276.44625213, -279.34299554, ...,  180.66066588,
         121.10979817, -288.03322576]),
 array([-281.79748804, -201.92697517, -146.77828771, ..., -281.79748804,
        -283.69916692, -283.69916692]),
 array([-274.22631793, -253.27230487, -248.61585753, ..., -269.56987059,
        -269.56987059, -255.60052855]),
 array([-263.2371022 , -258.25470354, -248.28990621, ..., -243.30750755,
        -263.2371022 , -248.28990621]),
 array([-260.39001725, -253.27230487, -256.83116106, ..., -260.39001725,
        -256.83116106, -260.39001725]),
 array([-268.21950086, -273.20189952, -273.20189952, ..., -263.2371022 ,
        -270.71070019, -270.71070019]),
 array([-272.64829967, -220.05631381, -214.52031529, ..., -275.41629892,
        -275.41629892, -278.18429818]),
 array([-271.95629985, -271.95629985, -271.95629985, ..., -271.95629985,
        -268.84230069, -271.95629985])]

[loli]

This might well be the expected behavior. To avoid information loss, the method stretches the intensity range. There's no mechanism to force a passive range. If you require positive values, you can simply shift them by the global min.

You might want to take a look if the resulting transformed images look correct. Is that the case?

And are the original image values all positive or bound between 0 and 255? What are the global min and max values over ask images? I don't remember how Bjorn published his data right now. Although this doesn't really matter.

[lazypoet]

Though the original image has a far greater range than just 0-255, the mask that I used ensures that the data is positive(you can see that in the code). Also, why would the method stretch it into negative if you are using auto? But yes, the transformed images do look okay.

[loli]

You are using the L4 percentile model. It's defined as L4 = [10, 20, 30, 40, 50, 60, 70, 80, 90]

The linear slope between the 10 and 20 percentile is applied to the lower outlier values. If it is steep enough, this might result in negative values.

At this short glance into the code I can't be sure, but you might achieve the desired effect with the model

[0, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100]

If that makes sense I can't say.

[lazypoet]

Okay, but listen to this. I was using SimpleITK to read the images. As you might know, for some reason(high dynamic range?), the images read by SimpleITK have intensity values in the range of upto a few thousands. Now, when I ran your method on those, the negative values go as much as -3000 or even more. I don't know if the slope can be that much