Create a new hdf5 with augmented patches

[Rob174]

Main ideas:

Problems: Currently f4290b7 :

• generating patches is too slow (especially loading large images into ram memory.
• GPU is largelly underused
• CPU is overloaded by large arrays preprocessing
• Training on 100 epochs takes around 1 day 10h (with the risk of memory error if the ram is also used by other programs)

Solution: Put a balanced augmented dataset patches of 1000 px width (square).

Problem: We would like to have rotated source images on which we compute patches. Too much memory required

Solution: Inverse transformation

[Rob174]

Rotation optimization: inverse transformation

Goal: avoid to load all source image into memory when we only want part of a rotated image

[Rob174]

More simple solution:

1. "Make" the rotation transformation (we do not make it but integrate the transformation inside the affine transformation)
2. center the patch by adding a shift
3. Use the dsize argument to crop the desired zone

Question: does opencv compute all pixels position ?

[Rob174]

Test opencv method

We will test if thusing this method speedup the code. For that we will compare two method. We will build an image with a rotated black rectangle of a known shift of (10000,10000) (before a rotation of -45°). We want our algorithm to be able to get only this patch. To achieve that goal we will compare the execution time of two approaches Approach 1:

1. Construct the original rotation matrix of +45°
2. Add a shift of (10000,10000) +/- center
3. Extract with a dsize of 1000,1000

Approach 2:

1. Rotate the image of 45°
2. Slice the zone beginning at (10000,10000) with a size of 1000

[Rob174]

Note: getRotationMatrix2D of opencv2 compose 3 transformation matrices: it translate the center provided to 0,0, rotation and shift the image back

So to create our custom rotation matrix we will add another translation that will put the patch at the upper left corner of the image and we will take on output window of the size of the patch

Note: the destination window is computed taking xwindowsize... pixels and ywindowsize... from the upper left corner of the original full size window

[Rob174]

Working test in 70236f4

[Rob174]

Order in which we will compute image transformations:

[Rob174]

Pb: if we pick a random angle and resize factor how to create coherent patches without duplicates ?

[Rob174]

Manage augmentation parameters at the dataset level

Idea:

• Augment source image
• Extract all patches (with filtering)

Need to save augmentation to compute for an input image

[Rob174]

Need to be saved at least at the Augmenter level as multiple classes manage augmentations computation

[Rob174]

We will include in the new Augmenter1 and RotationResizeMirrors the final resize (to a 256x256 image) and the mirrors: it will allow us to compute a big transformation matrix (that can be used to retrieve original coordinates with the inverse transformation)

[Rob174]

⚠️ ⚠️ opencv make a transformation matrix inversing traditionnal order: It is done such that to compute the transformation we do transformation_matrix.dot([cols,rows,1])

[Rob174]

After a profiling we can notice that the read step takes the most time (9938/4 ms = 2484.5 ms ~ 2.5s/call) We cannot avoid that as opencv require a numpy array as input for the transformations

[Rob174]

Stats: 1 epoch = 49 usable patches = 27min15s -> 33s between 2 usable patches in general 180 patches usable with cache creation script (for one set of augmentations)

[Rob174]

For further usage, to create the cache use the cache_generation branch

[Rob174]

Cf #29 for bug informations

[Rob174]

Done and checked in 3636648