how to implement Feature level Cutmix?

[emma-sjwang]

Thanks for your sharing.

But I have a question about how to implement 'ResNet-50 + Feature Cutmix', since if we just use the replace operation like we did in the image level cutmix, the gradients will not be back propagated.

Actually, I encountered such problem:

RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation

Could you give me some suggestions or hints to solve this feature level cutmix problem?

[hellbell]

@EmmaW8 In our implementation, we put this cutmix operation into forward function of resnet.py model. For example, Feature CutMix after layer 1 is,

            ...
            x = self.layer1(x)

            # Feature CutMix
            bbx1, bby1, bbx2, bby2 = rand_bbox(x.size(), lam)
            x[:,:,bbx1:bbx2,bby1:bby2] = x[rand_index,:,bbx1:bbx2,bby1:bby2]

            x = self.layer2(x)
            ...

From our experience, inplace=True option may cause the runtime error, so try with inplace=False.

Thanks!

[emma-sjwang]

Thank you for your reply. Do you know more about how to implement inplace=False. Exactly, the runtime error is derived from the gradient calculation for the assignment operation.

[hellbell]

@EmmaW8 In my case, it was worked when I change nn.ReLU(inplace=True) to nn.ReLU(inplace=False). I hope this would work. :)

[zs-zhong]

Hello, thanks for your great job! I have a question about Feature Cutmix: When training with Feature-Level Cutmix, how is the label transformation? Keeping it the same with Image-Level Cutmix or no transformation on the label?

[hellbell]

@zzs1994

I have a question about Feature Cutmix: When training with Feature-Level Cutmix, how is the label transformation? Keeping it the same with Image-Level Cutmix or no transformation on the label?

We do it in the same way of image-level CutMix.