Feature tiling

[RyanDoesMath]

Tiling

This pull request introduces two new functionalities to the image processing library:

Description of the Two New Functions

1. tile_image: This function tiles an image into a grid of overlapping sub-images. This technique is useful for object detection models that struggle to detect small objects within a larger image. By creating these sub-images (tiles), the model can potentially better detect small objects that might be missed in the full image.
2. tile_annotations: This function complements tile_image and takes a list of bounding boxes or keypoints (refer to BoundingBox and Keypoint classes) along with image tiling parameters. It assigns each annotation to the tiles that completely enclose it. This allows for associating annotations with the corresponding tiles created from the original image.

This pull request includes necessary validation checks for input parameters to ensure proper usage.

How to use these functions:

from PIL import Image

# Assuming you have an image loaded as 'image'
# Set tiling parameters (adjust as needed)
slice_width = 256
slice_height = 256
horizontal_overlap_ratio = 0.1  # 10% horizontal overlap
vertical_overlap_ratio = 0.2  # 20% vertical overlap

# Get a list of tiled sub-images
tiles = tile_image(image, slice_width, slice_height, horizontal_overlap_ratio, vertical_overlap_ratio)

# Assuming you have a list of bounding boxes 'annotations'
tiled_annotations = tile_annotations(annotations, image.width, image.height, slice_width, slice_height, horizontal_overlap_ratio, vertical_overlap_ratio)

# Now 'tiles' is a list of PIL Image objects representing sub-images

Annotation Data Classes: BoundingBox and Keypoint

This pull request also introduces two new classes, BoundingBox and Keypoint, to the image processing library. These classes facilitate working with object detection data, specifically for tasks involving bounding boxes and keypoints.

The BoundingBox Class

Represents a bounding box around an object in an image.

• Attributes:
  • category (str): The category of the object within the bounding box.
  • left (float): The x-coordinate of the top-left corner of the bounding box.
  • top (float): The y-coordinate of the top-left corner of the bounding box.
  • right (float): The x-coordinate of the bottom-right corner of the bounding box.
  • bottom (float): The y-coordinate of the bottom-right corner of the bounding box.
• Constructors:
  • from_yolo(yolo_line: str, image_width: int, image_height: int, int_to_category: Dict[int, str]):
    • Constructs a BoundingBox from a line in a YOLO formatted labels file.
  • from_coco(coco_annotation: Dict, categories: List[Dict]):
    • Constructs a BoundingBox from an annotation in a COCO data JSON file.
• Properties:
  • center (Tuple[float]): A tuple containing the (x, y) coordinates of the bounding box's center.
  • box (List[int]): A list containing the bounding box coordinates as [left, top, right, bottom].
• Methods:
  • to_yolo(image_width: int, image_height: int, category_to_int: Dict[str, int]) -> str: Writes a yolo formatted string using this bounding box's data.
  • validate_box_values(cls, left: float, top: float, right: float, bottom: float) -> None: Validates the box parameters and throws a value error if left > right or top > bottom. Also issues a warning for the case when left == right or top == bottom.

The Keypoint Class:

Represents a keypoint-boundingbox pair associated with an object in an image. These are used only for blood pressure and heart rate on the charts at the time of this writing.

• Attributes:
  • keypoint (Tuple[float]): A tuple containing the (x, y) coordinates of the keypoint relative to the top-left corner of the image.
  • bounding_box (BoundingBox): A BoundingBox object that defines the bounding box around the object containing the keypoint.
• Constructors:
  • from_yolo(yolo_line: str, image_width: int, image_height: int, id_to_category: Dict[int, str]):
    • Constructs a Keypoint from a line in a YOLO formatted labels file (ignores visibility information).
• Properties:
  • category (str): The category of the object the keypoint belongs to (inherited from the bounding_box).
  • center (Tuple[float]): The (x, y) coordinates of the bounding box's center (inherited from the bounding_box).
  • box (Tuple[float]): A list containing the bounding box coordinates as [left, top, right, bottom] (inherited from the bounding_box).
• Methods:
  • to_yolo(self, image_width: int, image_height: int, category_to_id: Dict[str, int]) -> str:
    • Generates a YOLO formatted string representation of this Keypoint object.
  • validate_keypoint(cls, bounding_box: BoundingBox, keypoint: Point) -> None:
    • Validates that a keypoint lies within the specified bounding box.

These classes provide a foundation for working with object detection data in various formats. They offer functionalities to parse common object detection data formats (YOLO, COCO), validate data integrity, and generate output in the desired format. Any place that bounding boxes are used should from now on implement a new constructor in BoundingBox and create BoundingBox objects to work with (this will happen with the outputs of computer vision models when that feature is added).