How to augment an image with channel number larger than 3?

[somebodyus]

How to augment an image with channel number larger than 3? Specifically, I have an RGB-D image. The channel number is 4.

[Dipet]

The simplest way to do this is to use 4 channel image with shape [Height, Width, 4]. Also you could use additional targets for this purpose https://albumentations.ai/docs/examples/example_multi_target/ But keep in mind that the additional channel must be the same height and width.

[yu-Mas]

The simplest way to do this is to use 4 channel image with shape [Height, Width, 4].

Hello,i used the 4 channel image(numpy) just like (256,256,4), but it still made some errors, just like 'F.shift_hsv' will raise 'ValueError: cannot reshape array of size 196608 into shape (256,256,4)'. Thanks!

[BloodAxe]

Well, I wonder what did you expect to get as a result? Some augmentations, require rgb input, which is clearly stated in the documentation. Second, your have different modalities in your RGB-D image. Albumentations does not support depth map augmentation as of now. However you can treat your depth map as a mask, and augment it as follows:

result = transform(image=input[..., :3], mask=input[...,3])

This will give you consistent spatial augmentation of the depth map. Note that interpolation mode for masks is hard-coded to "nearest neighbor".

[yu-Mas]

Well, I wonder what did you expect to get as a result? Some augmentations, require rgb input, which is clearly stated in the documentation. Second, your have different modalities in your RGB-D image. Albumentations does not support depth map augmentation as of now. However you can treat your depth map as a mask, and augment it as follows:

result = transform(image=input[..., :3], mask=input[...,3])

This will give you consistent spatial augmentation of the depth map. Note that interpolation mode for masks is hard-coded to "nearest neighbor".

Thanks for your patiently reply！I get it！

[somebodyus]

Well, I wonder what did you expect to get as a result? Some augmentations, require rgb input, which is clearly stated in the documentation. Second, your have different modalities in your RGB-D image. Albumentations does not support depth map augmentation as of now. However you can treat your depth map as a mask, and augment it as follows:

result = transform(image=input[..., :3], mask=input[...,3])

This will give you consistent spatial augmentation of the depth map. Note that interpolation mode for masks is hard-coded to "nearest neighbor".

If the depth map has already be normalized to 0-255, can we directly transform the 4-channel RGBD image? result = transform(image=input[..., :4], mask=mask)

[Dipet]

Yes you can, but some transforms like RGBShift, HueSaturationValue only work with RGB images.

[JJrodny]

Couldn't we modify albumentations to augment the RGB channels only for 4 channel images (RGBA and RGB-D?)

Like

def apply(self, image, hue_shift=0, sat_shift=0, val_shift=0, **params):
    if not is_rgb_image(image) and not is_grayscale_image(image) and not is_rgba_image(image):
        raise TypeError("HueSaturationValue transformation expects 1-channel, 3-channel, or 4-channel images.")
    if is_rgba_image(image):
         return np.concatenate((F.shift_hsv(image[0:3], hue_shift, sat_shift, val_shift), image[3]), axis=0)
    else:
         return F.shift_hsv(image, hue_shift, sat_shift, val_shift)

instead of

def apply(self, image, hue_shift=0, sat_shift=0, val_shift=0, **params):
    if not is_rgb_image(image) and not is_grayscale_image(image):
        raise TypeError("HueSaturationValue transformation expects 1-channel or 3-channel images.")
    return F.shift_hsv(image, hue_shift, sat_shift, val_shift)

here: https://github.com/albumentations-team/albumentations/blob/e3b47b3a127f92541cfeb16abbb44a6f8bf79cc8/albumentations/augmentations/transforms.py#L1005C21-L1005C21

I created a comment about it here, but found this discussion after deeper google searching

[yu-Mas]

  您好，邮件已收到，我会尽快处理，谢谢。