What is the best way of implementing a custom augmentation requiring both images and bounding boxes?

[rex-yue-wu]

Hi all,

I need to implement a customized crop-like data augmentation where the cropping parameters need to be randomly drawn based on bounding boxes information with some constraints.

In short, this is how to do it when given a pair of image and bounding boxes with classes.

def get_random_crop_paramerters(bounding_boxes,
                                prng,
                                min_box_size=0.1, 
                                min_num_bboxes=1, 
                                min_aspect_ratio=0.5, 
                                max_aspect_ratio=0.5, 
                                max_trials=50) :
    """get random crop parameters meeting the given constraings 
    """
    num_trials = 1
    crop_params = None
    while num_trials <= max_trails :
        crop_x0, crop_y0, crop_x1, crop_y1 = prng.uniform(size=(4,))
        if meet_constraints( crop_x0, crop_y0, crop_x1, crop_y1, bounding_boxes, 
                            min_box_size, 
                            min_num_bboxes,
                            min_aspect_ratio=0.5, 
                            max_aspect_ratio=0.5,
                           ) :
            crop_params = [crop_x0, crop_y0, crop_x1, crop_y1]
            break
        else :
            num_trails +=1 
    if crop_params is None :
        crop_params = [0, 0, 1, 1]
    return crop_params

def augment_one_sample(image, bounding_boxes, prng=np.random.RandomState(None)) :
    """augment a single sample
    """
    crop_params = get_random_crop_paramerters(bounding_boxes, prng)
    cropped_image = apply_crop_to_image(image, crop_params)
    cropped_bounding_boxes = apply_crop_to_bounding_boxes(bounding_boxes, crop_params)
    return cropped_image, cropped_bounding_boxes

I checked the lambda function, but it seems that images and bounding boxes need to be done separately. Not sure how to pass the potential cropping parameters back to the image augmentation function.

Can anyone help?

[b04505009]

+1 I also have the same question.