LinasKo
commented
3 months ago I'm bringing the SlicerBlock code I had up-to-speed.
@grzegorz-roboflow, with respect to your comments:
- We can't change the params to InferenceSlicer as it is in
supervision, unfortunately. We can, however, wrap the implementation with custom logic, if that makes it more intuitive for the users. Do you think we should do that? - I've removed the back-and-forth conversion. In this case, when
sv.Detectionsare obtained, they'll be kept. - I've added an integration test for the slicer, using an image from
assets.
@grzegorz-roboflow or @PawelPeczek-Roboflow, would you have time to take a look at this?
PawelPeczek-Roboflow
Description
This PR adds the
RoboflowDetectionSlicerBlockandRoboflowSegmentationsSlicerBlockblocks and respective unit tests, but not integration tests. If needed, I can add those after the lighthouse.They can be found in the new folder, under
inference/core/workflows/core_steps/supervision_tools/.This is a draft PR as I want to address a few items mentioned in the
Review suggestionssection below.Type of change
How has this change been tested, please provide a testcase or example of how you tested the change?
mkdocs serve, observing that blocks appear in the blocks list.simulate_detections_slicer_block.py
```python import os import cv2 import numpy as np import supervision as sv from inference.core.managers.base import ModelManager from inference.core.registries.roboflow import ( RoboflowModelRegistry, ) from inference.core.workflows.core_steps.supervision_tools.detections_slicer import RoboflowDetectionSlicerBlock from inference.core.workflows.entities.base import Batch, ImageParentMetadata, WorkflowImageData from inference.models.utils import ROBOFLOW_MODEL_TYPES API_KEY = os.getenv("ROBOFLOW_API_KEY") assert API_KEY is not None, "Please set the ROBOFLOW_API_KEY environment variable" model_registry = RoboflowModelRegistry(ROBOFLOW_MODEL_TYPES) model_manager = ModelManager(model_registry=model_registry) block = RoboflowDetectionSlicerBlock( model_manager=model_manager, api_key=API_KEY ) img_path = "cat.png" image = cv2.imread(img_path) # image = np.zeros((192, 168, 3), dtype=np.uint8) images = Batch([ WorkflowImageData( parent_metadata=ImageParentMetadata(parent_id="$inputs.image"), numpy_image=image, ) ]) async def main(): model_id = "yolov8n-640" box_annotator = sv.BoundingBoxAnnotator() label_annotator = sv.LabelAnnotator() res = await block.run_locally( images=images, model_id=model_id, class_agnostic_nms=None, class_filter=None, confidence=0.4, iou_threshold=0.4, slice_width=640, slice_height=640, overlap_ratio_width=0.2, overlap_ratio_height=0.2 ) detections = res[0]["predictions"] ann = image.copy() ann = box_annotator.annotate(ann, detections) ann = label_annotator.annotate(ann, detections) cv2.imwrite("det_local.png", ann) print(f"#####################\nLocal run result:") print(res) res = await block.run_remotely( images=images, model_id=model_id, class_agnostic_nms=None, class_filter=None, confidence=0.4, iou_threshold=0.4, slice_width=640, slice_height=640, overlap_ratio_width=0.2, overlap_ratio_height=0.2 ) detections = res[0]["predictions"] ann = image.copy() ann = box_annotator.annotate(ann, detections) ann = label_annotator.annotate(ann, detections) cv2.imwrite("det_remote.png", ann) print(f"#####################\nRemote run result:") print(res) if __name__ == "__main__": import asyncio asyncio.run(main()) ```simulate_segmentations_slicer_block.py
```python import os import cv2 import numpy as np import supervision as sv from inference.core.managers.base import ModelManager from inference.core.registries.roboflow import ( RoboflowModelRegistry, ) from inference.core.workflows.core_steps.supervision_tools.segmentations_slicer import RoboflowSegmentationSlicerBlock from inference.core.workflows.entities.base import Batch, ImageParentMetadata, WorkflowImageData from inference.models.utils import ROBOFLOW_MODEL_TYPES model_registry = RoboflowModelRegistry(ROBOFLOW_MODEL_TYPES) model_manager = ModelManager(model_registry=model_registry) block = RoboflowSegmentationSlicerBlock( model_manager=model_manager, api_key=os.getenv("ROBOFLOW_API_KEY") ) img_path = "cat.png" image = cv2.imread(img_path) images = Batch([ WorkflowImageData( parent_metadata=ImageParentMetadata(parent_id="$inputs.image"), numpy_image=image, ) ]) async def main(): model_id = "yolov8n-seg-640" mask_annotator = sv.MaskAnnotator() label_annotator = sv.LabelAnnotator() res = await block.run_locally( images=images, model_id=model_id, class_agnostic_nms=None, class_filter=None, confidence=0.2, mask_decode_mode="accurate", tradeoff_factor=0.0, iou_threshold=0.2, slice_width=640, slice_height=640, overlap_ratio_width=0.2, overlap_ratio_height=0.2 ) detections = res[0]["predictions"] ann = image.copy() ann = mask_annotator.annotate(ann, detections) ann = label_annotator.annotate(ann, detections) cv2.imwrite("seg_local.png", ann) print(f"#####################\nLocal run result:") print(res) res = await block.run_remotely( images=images, model_id=model_id, class_agnostic_nms=None, class_filter=None, confidence=0.2, mask_decode_mode="accurate", tradeoff_factor=0.0, iou_threshold=0.2, slice_width=640, slice_height=640, overlap_ratio_width=0.2, overlap_ratio_height=0.2 ) detections = res[0]["predictions"] ann = image.copy() ann = mask_annotator.annotate(ann, detections) ann = label_annotator.annotate(ann, detections) cv2.imwrite("seg_remote.png", ann) print(f"#####################\nRemote run result:") print(res) if __name__ == "__main__": import asyncio asyncio.run(main()) ```Review suggestions
Here's a few areas that need to be looked at:
RoboflowSegmentationsInferenceSlicer,RoboflowDetectionsInferenceSlicerare a mouthful.supervision_tools.segmentations_slicer.pyanddetections_slicer.py.segmentations_slicer.py, when"points"are set andsv.mask_to_polygonsis called, I've selected the first polygon. I need to think whether that's always the case.Any specific deployment considerations
Docs
Two new blocks are present in the Blocks list. CTRL+F for
slicer.