Consider adding a denoising filter before segmentation

[jcohenadad]

Some denoising filters are quite powerful, and could potentially help during model training/testing. Eg:

sct_maths -i sub-P001_UNIT1.nii.gz -denoise 1 -o sub-P001_UNIT1_denoised.nii.gz 

Few issues though:

• this algorithm is slow (several minutes on a large volume). So I guess that wouldn't work when applying it during DA. But maybe we could add this step during preprocessing? Or maybe we can only apply this step during inference?
• if the denoising is too aggressive, it can "remove" a small lesion.

One way to mitigate the two arguments above is to use a less aggressive denoising filter kernel.

[uzaymacar]

Admittedly without a lot of experience with denoising, I would be in favor of introducing it during data augmentation in training to help the model generalize to noised and de-noised images. It makes more sense to leave the decision whether to apply denoising or not beforehand during inference to the user as denoising can be helpful or harmful depending on the image content.

We have already implemented random blur in ivadomed and use it in this project as a smoothing operator. If the goal of the denoising mentioned here is to smooth + preserve edge-information, we could perhaps implement bilateral filtering in ivadomed as well (which I'm guessing can be faster than sct_maths -denoise)?

[jcohenadad]

blurring is quite different than denoising, and IIUC bilateral filtering is a vanilla of blurring. I could be wrong thought, so maybe worth investigating? (esp if, as you said, it should be way faster than sct_maths denoise)

[uzaymacar]

I think this is an interesting discussion -- I will try to investigate further and update this issue with results (trade-off between speed and performance across several algorithms)! For now, I am reading a review on image denoising techniques and it mentions the following:

The major challenges for image denoising are as follows: (a) flat areas should be smooth, (b) edges should be protected without blurring, (c) textures should be preserved, and (d) new artifacts should not be generated.

Bilateral filtering [10] is widely used for image denoising.

A pioneering work on non-local means (NLM) [38] used the weighted filtering of the NSS prior to achieve image denoising, which is the most notable improvement for the problem of image denoising.

I see that SCT uses a version of the non-local means (NLM) from the documentation of SCT.

[uzaymacar]

For a 3D volume (from sub-P001) of size (240, 256, 176), I observed the following runtimes on my laptop:

Denoising method	Runtime (~)
NLM with block_radius=5 (default in SCT)	500 secs
NLM with block_radius=3	90 secs
NLM with block_radius=2	25 secs
Bilateral filter	200 secs

Insights about the table:

• For comparison, gaussian smoothing / blurring is a common data augmentation and it takes ~1 sec to run on my laptop.
• For NLM, I used the implementation in SCT.
• Bilateral filtering was implemented using SimpleITK, specifically the simpleITK.BilateralImageFilter() class
• Different block_radius values for NLM were tried to see how much time can we gain.

Denoising algorithms are very slow to the extent that we can't introduce them as data augmentations which supports @jcohenadad's suggestion. Yet, we can still make NLM a lot faster by reducing the block_radius without a lot of difference in the output.

I would like add this step as a new preprocessing transformation for ivadomed if possible. This will allow us to apply denoising directly within the config file and faster experiment to conclude whether denoising can help or not during training and / or testing. @jcohenadad would this be OK? Asking because this will introduce some overlap between ivadomed and SCT.

[jcohenadad]

I would like add this step as a new preprocessing transformation for ivadomed if possible. This will allow us to apply denoising directly within the config file and faster experiment to conclude whether denoising can help or not during training and / or testing. @jcohenadad would this be OK? Asking because this will introduce some overlap between ivadomed and SCT.

I'm totally OK with this!

[jcohenadad]

I did a bit of investigations, and found good parameters:

sct_maths -i sub-P001_UNIT1_crop.nii.gz -denoise p=1,b=2 -o sub-P001_UNIT1_crop_denoised-p1b2.nii.gz 

Note: this should be run on the cropped image (for lesion segmentation).

[jcohenadad]

Can close as new GTs are now done