Extra Cutout Operation in Preprocessing

[sharonzhou]

This extra cutout operation is (deterministically for every image) done in the repository and not mentioned in the updated paper using RandAugment. Is the performance reported in the paper consistent with this implementation or the one specified in the paper?

https://github.com/google-research/uda/blob/6aabffa896458fb7806eba9d94fbfeca77f3b72e/image/preprocess.py#L274

More context:

 for image in ori_images:
    chosen_policy = aug_policies[np.random.choice(
        len(aug_policies))]
    aug_image = augmentation_transforms.apply_policy(
        chosen_policy, image)
    aug_image = augmentation_transforms.cutout_numpy(aug_image)

[michaelpulsewidth]

Hi, this is expected. Cutout is applied by default in RandAugment. In Section 4.2 of the RandAugment paper, it is mentioned that "The default augmentations for all methods include flips, pad-and-crop and Cutout".

[sharonzhou]

Yes, but this Cutout line is after the RandAugment operation. RandAugment (which includes Cutout, in addition to flips and pad-and-crop) was applied to the image beforehand in the preceding lines.

chosen_policy = aug_policies[np.random.choice(
       len(aug_policies))]
aug_image = augmentation_transforms.apply_policy(
       chosen_policy, image)

But then Cutout was applied (again, in some cases) after RandAugment. aug_image = augmentation_transforms.cutout_numpy(aug_image)

[michaelpulsewidth]

I think we might have a different understanding of "The default augmentations for all methods include flips, pad-and-crop and Cutout". Do you mean that Cutout, flip & pad-and-crop are in the search space of RandAugment? It actually means that they apply Cutout, flip, pad-and-crop in addition to two random operations.

This is similar to what has been done in AutoAugment:

      epoch_policy = self.good_policies[np.random.choice(
          len(self.good_policies))]
      final_img = augmentation_transforms.apply_policy(
          epoch_policy, data)
      final_img = augmentation_transforms.random_flip(
          augmentation_transforms.zero_pad_and_crop(final_img, 4))
      # Apply cutout
      final_img = augmentation_transforms.cutout_numpy(final_img)

So this is what has been done in AutoAugment: two operations->pad_and_crop->flip->cutout.

[sharonzhou]

Got it, thank you so much for your help. Closing this.

Our consistency loss was surely too low to be reasonable with a single random transformation..

[sharonzhou]

Though it would be awesome to clarify this in the manuscript :)

To clarify, the order of operations in this code base looks like: flip -> crop -> two random ops -> cutout

where flip and crop are preprocessing steps on CIFAR-10 https://github.com/google-research/uda/blob/1862464b44f6769864c27fc3a417723fb1e8ff2d/image/data.py#L65

where two random ops and cutout are in the file and line pasted above https://github.com/google-research/uda/blob/6aabffa896458fb7806eba9d94fbfeca77f3b72e/image/preprocess.py#L274

This would be awesome for reproducibility 😃

[michaelpulsewidth]

Yes. The order in this codebase should be two random ops -> cutout -> flip -> crop. The flip and crop is applied during training in data.py. We changed the order since we use random ops and cutout by numpy and PIL instead of building them into the TF computational graph. Flip and crop are applied during training so that the model sees more diverse images.