Offsets for extracting patches

idiap / attention-sampling

This Python package enables the training and inference of deep learning models for very large data, such as megapixel images, using attention-sampling

Other

98 stars 18 forks source link

Offsets for extracting patches #13

Open darleybarreto opened 4 years ago

darleybarreto commented 4 years ago

Hi, I have a couple of questions about offsets used to crop the original images:

An offset for an image I is [I, X, Y], where X varies on height and Y on width, correct?
What would mean an offset with a negative X or Y value at this line?

Thanks in advance!

angeloskath commented 4 years ago

Hi,

It means that the patch starts outside the original image which is assumed to be padded with zeros. So starting at a large negative offset for instance would guarantee a black output image. Similarly very large positive offsets also guarantee a black output image.

You can see the C++ code for extract_patches in ats/ops/extract_patches/extract_patches.cc. You can see in lines 108 and 109 that we check if a certain pixel is inside and then in line 119 we copy 0 if it is not inside.

Cheers, Angelos

darleybarreto commented 4 years ago

Are these off-image indices bounded? Suppose I want to make a bounding box from each offset, a value may be outside the padded ones So the best thing to do would be clipping them to the range [0, MAXSIZE], right?

angeloskath commented 4 years ago

Hm I am not sure I understand. What would the bounding box enforce? That the indices do not go out of the bbox? Then what does it mean to extract a patch centered at 0,0 with patch size 50,50? 0 to 50 or -25 to 25?

darleybarreto commented 4 years ago

The bounding box wouldn't enforce the offsets value range, I want them to be used on the original high dimensional images to perform cropping (for visualization purposes) and be able to compute some metrics (like IoU)