Image Augmentation for 3D images

[subhajitchatu]

Hi, Imgaug is a very good tool for generating images. I am trying to generate different data from my 3D MRI image data. As the code is only supporting 2D images or RGB images, imgaug is giving failure in case of my data. This might seem as my personal different problem, but I want your suggestion of how can we change the code so that it will support 3D images. My images are 25625660 sizes. Its a dicom image, and it has 60 slices. Generating augmented 2D image will not touch Z axis, which is not acceptable in my case. Please suggest something using which I can achieve the same feature, mainly I will use affine,piecewise affine and elastic transformation for generating new data.

Thanks

[aleju]

Are these images each numpy arrays of shape 256x256x60 with dtype uint8? Then you should be able to use affine/piecewise affine/elastic transformations. Those augmenters don't have any restrictions with regards to the numbers of channels per image and should augment each channel in the same way (per image). At 256x256x60 they will probably be quite slow, but should work. Most other augmenters should also work with 60 channels. Colorspace transformations and Grayscale() would crash though.

[subhajitchatu]

Thanks @aleju

dtype is not uint8, it is uint16. It can be float also. Yes I know it will be a slow process, but actually it will have great usage. Just think of CT scan,MRI, images, they all are taken in different slices for different cross section of a part. Basically in my case one MRI 3D Dicom image has 60 different slices and each slice is of 256*256 sizes. Image intensity can vary upto 2000. It will be great if imgaug can able to handle this requirement.

Thanks

[aleju]

You should stack the 60 256x256 images to one 256x256x60 image and try to apply the augmenters to that (and afterwards split them up again). I think the ones that you want to use should work with uint16 and float, though I never explicitly tried it.

[subhajitchatu]

Hi, @aleju It's working! Thanks. Frankly speaking I have to simulate some motions in these stack of 3D brain images. The motion is related to body motion or breathing motion or flow of blood motions.

These can be rigid motion or non-rigid motion. Rigid motions can be generated using affine transformation , but I am little confused in generating breathing motion, whose only constraint is some degree of continuity or smoothness. I am not sure if I can manage it using elastic motions. Piece-wise affine augmenter can somehow help in generating blood flow motions, as they will bring the local motions. But I am concerned about other deformations which violates both continuity or smoothness. Or motions where whole surface shrinks or expands. So I just want to know about different elastic,fluid or non-rigid motion generation using augmenters. Just think of motions in any part of the body, like breathing(vertical or horizontal depending upon the image cross section sagittal or coronary), motions of the patient(body shaking), motion due to flow of fluid , blood etc. Just throw some lights if you have idea in this kind of medical imaging motions.

Thanks

[aleju]

Elastic Transformations does not really do more than moving individual pixels around in a random fashion. That seems to me like it is most suited to simulate errors of the scanner, but not for rigid transformations. PiecewiseAffine is more suited for that. It basically places a regular grid of a few anchor points on the image (like 4x4 points), moves each point randomly around and then generates local affine transformations that match that movement of the points. The affine transformations are smooth between the points. The augmenter offers parameters for the scale (strength) of moving the points, for the number of rows in the grid and the number of columns. For your image sizes you might want to use a finer grid than 4x4, e.g. RandomAffine(scale=(0.0, 0.02), nb_rows=(4, 16), nb_columns=(4, 16)). If you want to get augmentations that have their "center" (highest strength) at a specific slice and become weaker towards the "outer" slices, then that would be quite a bit harder to do. You would have to initialize an augmenter per image, then copy that 60 times (so that all slices use the same grid) and manually change the scale value per slice to a specific value (so that it becomes smaller towards the outer slices). If you want to simulate shaking of the camera or the whole scanned object, you can probably do that using Affine with translation.

[subhajitchatu]

Thanks @aleju . Motions can be global rigid motions(can be handled with affine augmenters) or elastic deformation or bspline based local deformations. For the second one we don not have any augmentor. Typical example of nonrigid is deformation of muscle. Input will be an 3D image and output will be a newly generated data incorporated the desired nonrigid motion. There should be some mathematical formula or model, using which we could generate artificial motion data. But there is no standard formula, but I think it is possible to add some random bspline transform at different points on grid and generate local nonrigid motion OR we can use some force or energy gradient formula for simulating elastic deformations.

I was wondering if there is any package or software available to do this, but didn't find any for doing this nonrigid part.Actually if we can generate this, using this type of feature we can validate any registration methods or simulate different deformation models.

I want some help in generating such artificial data using python or matlab for NIFTI/DICOM 3D images. Please go through the below links for detail info.

Thanks

http://www.cs.toronto.edu/~jepson/researchNRM.html

page 24-45 https://www.slideshare.net/UlaBac/lec16-medical-image-registration-advanced-deformable-registration

[aleju]

Not sure I can help you there any further. I don't really know much about bspline transformations and currently don't have enough time to learn it. Though it still sounds to me like PiecewiseAffine should mostly do what you want.

[nazib]

Hi there Please take a look on this repository and generate as many deformation as you can by simply tuning parameters for 3D medical Images. There is a file named "deformation_test.py" which is the idea from Hessam Sookati. You can use that one.

[xuzhang5788]

@nazib I read your code which is named "deformation_test.py. In it, you comment out "#Z=np.reshape(Z,[1,mnc])" in your def kernel(). If my image data shape is (nbexamples, nx, ny, nz, nbchannels), is it possible to use your library? many thanks

[nazib]

Hi @xuzhang5788 My code is for deforming one 3D image at a time. I dont think its possible to deform a full batch of images. My method is not applicable in such case.

[techkang]

I think this code may work for you. Assume that your picture is DHW*C

seq = iaa.Sequential(...)
seq.deterministic = True
image = np.array([seq(image=i) for i in image])
seq.deterministic =  False