Find appropriate Data Augmentation Script and Apply it to the dataset (subset of dataset)

[AntonDahl]

Find some scripts for dataAug here: https://github.com/AgaMiko/data-augmentation-review/blob/master/README.md#Computer-vision

[rverschuren]

The interest of performing data augmentation for this project is double:

• Improving the performances of the model.
• Creating model abstraction of the colors and the textile density.

One way of measuring the second point is to apply anomaly detection on another type of textile and see how it can perform.

It is important to choose the right augmentation operations to avoid creating anomalies by augmenting the data.

Some ideas to implement

• [ ] Blurring: to simulate different densities of textile.
• [ ] Colors
• [ ] others...

If you want to run the model and output the augmentation to observe what it is doing:

Run the following command !python train_resnet.py --phase augment --category carpet --num_epochs 3

[rverschuren]

Using this library : https://github.com/kornia/kornia

And this example https://pytorch-lightning.readthedocs.io/en/stable/notebooks/lightning_examples/augmentation_kornia.html

[rverschuren]

Default

Code:

        self.data_transforms = transforms.Compose([
                        transforms.Resize((args.load_size, args.load_size), transforms.InterpolationMode.LANCZOS),
                        transforms.ToTensor(),
                        transforms.CenterCrop(args.input_size),
                        transforms.Normalize(mean=mean_train,
                                            std=std_train)])

Image:

Default mode but without normalization and centercrop.

Code:

        self.data_transforms = transforms.Compose([
                        transforms.Resize((args.load_size, args.load_size), transforms.InterpolationMode.LANCZOS),
                        transforms.ToTensor()    ])

Image:

[rverschuren]

If you want to add some augmentations.

1. Import it by adding the class in the import line from kornia.augmentation import RandomBoxBlur, ColorJitter, Normalize [OTHER.....]

2. Add in the sequential composition of augmentations:

   class DataAugmentation(nn.Module):
   """Module to perform data augmentation using Kornia on torch tensors."""
   
   def __init__(self, apply_color_jitter: bool = False) -> None:
       super().__init__()
       self._apply_color_jitter = apply_color_jitter
   
       self.transforms = nn.Sequential(
           RandomBoxBlur(kernel_size=(3, 3), border_type='reflect', p=0.75),
           Normalize(mean=mean_train, std=std_train)
           [ADD HERE]
       )

3. Run to observe output files: !python train_resnet.py --phase augment --category carpet --num_epochs 3

4. open the 2 generate files.

   "img_augment.png" "img_before_augment.png"

[rverschuren]

I tried to add the RandomRotation, but the problem is that we lose information on the image and a part of the image is set to black pixels. Which could be detected as an anomaly. By using the RandomAffine(degrees=45.0, scale=(1,2), padding_mode=2, p=.75), we can set the scale value and the padding_mode which allows us to keep the texture of the textile everywhere on the image.

Se the difference of RandomRotation and RandomAffine on the newt two images.

[rverschuren]

Now let's compare when playing with colors, contrast, brightness, ...

Using the kornia.augmentation.ColorJiggle class.

To better understand the difference, I did not normalize the data before saving the image. The two next images are

• with brightness=0.1, contrast=0.1, saturation=0.1, hue=0.1

Without normalization With normalization

• with brightness=0.2, contrast=0.2, saturation=0.2, hue=0.2

  Without normalization With normalization

In the commit I set it to 0.1

[rverschuren]

Adding some blur to obtain something similar to leather.

        self.transforms = nn.Sequential(
            RandomBoxBlur(kernel_size=(2,2), border_type='reflect', p=0.2),
            RandomBoxBlur(kernel_size=(3,3), border_type='reflect', p=0.2),
            RandomBoxBlur(kernel_size=(5,5), border_type='reflect', p=0.2),
            RandomBoxBlur(kernel_size=(7,7), border_type='reflect', p=0.2),
            RandomBoxBlur(kernel_size=(5,5), border_type='reflect', p=0.2),
            RandomBoxBlur(kernel_size=(15,15), border_type='reflect', p=0.1),
            RandomAffine(degrees=45.0, scale=(1,2), padding_mode=2, p=.75),
            ColorJiggle(0.1, 0.1, 0.1, 0.1, p=1.),
            #Normalize(mean=mean_train, std=std_train)
        )