search

pytorch / vision

Datasets, Transforms and Models specific to Computer Vision
https://pytorch.org/vision
BSD 3-Clause "New" or "Revised" License
16.29k stars 6.96k forks source link

Feature proposal: Cutout #420

Open juneoh opened 6 years ago

juneoh commented 6 years ago

Improved Regularization of Convolutional Neural Networks with Cutout proposes a simple and effective method for regularization. I think it would fit greatly in the torchvision.transforms toolbox, and am willing to contribute an implementation with a proper test, if that's okay.

alykhantejani commented 6 years ago

@juneoh is this a similar proposal to the one in #226?

juneoh commented 6 years ago

@alykhantejani It is indeed similar, but the key difference is that the cited Cutout paper has successfully parameterized effective random occlusions: e.g. the shape, be it polygons to free shapes, matters less than the size, so square patches are sufficient; allowing portions of the occlusion to lie outside the image is critical to good performance; etc. There need only be two hyperparameters for an implementation, the patch size and the fill value.

juneoh commented 6 years ago

Oh, but the paper is cited in #226. Perhaps I should have requested a reopen of that issue instead. I thought I've searched thoroughly enough to avoid a duplicate.

alykhantejani commented 6 years ago

cc @fmassa who closed out the previous issue - who might have some other thoughts on this

hendrycks commented 6 years ago

Both Cutout and Random Erasure are highly similar. If I recall correctly, Random Erasure has more parameters, though these parameters can be left to the default setting as is done in the paper. In contrast, Cutout has fewer parameters and is easier to describe, and contrariwise Cutout often requires more tuning.