Autoscale intensity

[DirectX]

Thanks for great project. Found one problem - input images in datasets usually have quite inconsistent input values range.

Here is small piece of code to get input values range stats for given dataset

import os
import numpy as np
import nibabel as nib

dirs = ['/path/to/datasets/FLARE2021/imagesTr']

for dir in dirs:
    filenames = os.listdir(dir)
    min_vals = []
    max_vals = []
    for filename in filenames:
        img = nib.load(os.path.join(dir, filename))
        data = img.get_fdata()
        min_vals.append(np.min(data))
        max_vals.append(np.max(data))
    print(f'  min:\n    min: {np.min(min_vals)}, median: {np.median(min_vals)}, avg: {np.average(min_vals)}, max: {np.max(min_vals)}')
    print(f'  max:\n    min: {np.min(max_vals)}, median: {np.median(max_vals)}, avg: {np.average(max_vals)}, max: {np.max(max_vals)}')
    print('\n------------------------------------------------------------------------------------------------------------\n')

Here is output for FLARE2021 training set

  min:
    min: -2048.0, median: -1024.0, avg: -1059.235457063712, max: -1024.0
  max:
    min: 1141.0, median: 1666.0, avg: 2025.0193905817175, max: 4009.0
------------------------------------------------------------------------------------------------------------

As you can see there is tremendous variability in input ranges across all images in dataset. On the other side there is hard-coded scaling parameters in preparation pipeline

...
elif dataset == 'flare':
        train_transforms = Compose(
            [
                ...
                ScaleIntensityRanged(
                    keys=["image"], a_min=-125, a_max=275,
                    b_min=0.0, b_max=1.0, clip=True,
                ),
                ...
            ]
        )
...

As a result of clipping of the input data values either to 0 or to 1 processed images could severely suffer from 'overexposure' and 'underexposure' making merely impossible to detect features by neural network.

This PR proposes approach like 'auto levels' feature in Photoshop.

It is not supposed to be merged AS-IS because it may not work for pre-trained weights but it could be taken into the account for new datasets training.

[leeh43]

Thank you to your interest about our work. I agree with your statement that each image has different scale of intensities due to different imaging protocols. However, the one that we perform is a soft-tissue windowing (clip the intensity range from -125 to 275), which use to minimize the adverse effect brings from the neighboring tissues and a must-do preprocessing step for abdominal imaging. From your autoscaling code, it only gives me a scaling function to normalize the intensity from 0 to 1 from the original intensity range.