Reduce training time by storing clusters of shapes only

Rob174 / detection_nappe_hydrocarbures_IMT_cefrem

0 stars 0 forks source link

Reduce training time by storing clusters of shapes only #27

Closed Rob174 closed 3 years ago

Rob174 commented 3 years ago

Observation

For 100 epochs with only patch augmentation 1 day 16h necessary
We waste a lot of time opening patches not containing interesting classes (seep, spill)

Potential solutions:

Extract seep and spill zones into a hdf5 cache and take that as new images
Resize input image so that patches are already at the correct size for the model

Rob174 commented 3 years ago

Pb:

min area rectangle stats computed in #22 show that most of the oil discharges are smaller than 500 px

Rob174 commented 3 years ago

Make cluster of shapes, use barycenters, use max distance between group of shape

Algo:

Make cluster of shapes by gathering shapes distant from less than ... px
Make patch(es) from each cluster

Rob174 commented 3 years ago

Pb: how to estimate minimal distance between shapes (represented as lists of points)

Rob174 commented 3 years ago

An edge between two point can be define in the following way:

$\begin{cases} \lambda_1 x_1 + (1-\lambda_1 )x_2\\ \lambda_1 y_1 + (1-\lambda_1 )y_2 \end{cases}$

Example:

Animation

We want to find the minimal distance between all possible of points on two edges from two shapes:

With this representation we can then compute the minimal distance between two points on each pair of edges possible

(Example with two edges possible)

To compute the minimal distince we will use scipe optimize minimize

Rob174 commented 3 years ago

Statistics number of shapes per images:

{'num_of_shape_on_image=1': 'num_of_images=20', 'num_of_shape_on_image=2': 'num_of_images=16', 'num_of_shape_on_image=3': 'num_of_images=20', 'num_of_shape_on_image=4': 'num_of_images=17', 'num_of_shape_on_image=5': 'num_of_images=8', 'num_of_shape_on_image=6': 'num_of_images=9', 'num_of_shape_on_image=7': 'num_of_images=10', 'num_of_shape_on_image=8': 'num_of_images=6', 'num_of_shape_on_image=9': 'num_of_images=4', 'num_of_shape_on_image=10': 'num_of_images=4', 'num_of_shape_on_image=11': 'num_of_images=3', 'num_of_shape_on_image=12': 'num_of_images=1', 'num_of_shape_on_image=13': 'num_of_images=1', 'num_of_shape_on_image=15': 'num_of_images=4', 'num_of_shape_on_image=16': 'num_of_images=2', 'num_of_shape_on_image=17': 'num_of_images=3', 'num_of_shape_on_image=20': 'num_of_images=2', 'num_of_shape_on_image=21': 'num_of_images=1'}

Rob174 commented 3 years ago

As we will use constant transformations with patches of fixed size, this issue will be closed and put into the corresponding column Final commit for the code f4290b7