InferenceSlicer : xyxy must be 2d np.ndarray with (n, 4) shape

[mwitiderrick]

Search before asking

• [X] I have searched the Supervision issues and found no similar bug report.

Bug

Getting the following error with InferenceSlicer when using from_deepsparse with a video frame but it works with a normal image

ValueError  Traceback (most recent call last)
[<ipython-input-23-d7959e19d363>](https://localhost:8080/#) in <cell line: 20>()
     18 slicer = sv.InferenceSlicer(callback = callback)
     19 
---> 20 detections = slicer(frame)
     21 detections

5 frames
[/usr/local/lib/python3.10/dist-packages/supervision/detection/core.py](https://localhost:8080/#) in _validate_xyxy(xyxy, n)
     19     is_valid = isinstance(xyxy, np.ndarray) and xyxy.shape == (n, 4)
     20     if not is_valid:
---> 21         raise ValueError("xyxy must be 2d np.ndarray with (n, 4) shape")
     22 
     23 

ValueError: xyxy must be 2d np.ndarray with (n, 4) shape

Environment

pip install deepsparse[yolo] supervision

Minimal Reproducible Example

from pytube import YouTube
youtube = YouTube('https://youtu.be/ItDDnNJz6JY')
video = youtube.streams.get_highest_resolution()
video.download()

import supervision as sv
import numpy as np
from deepsparse import Pipeline
VIDEO_PATH = "Pexels.mp4"

# extract video frame
generator = sv.get_video_frames_generator(VIDEO_PATH)
iterator = iter(generator)
frame = next(iterator)

model_stub = "zoo:cv/detection/yolov5-l6/pytorch/ultralytics/coco/base-none"
yolo_pipeline = Pipeline.create(task="yolo", model_path=model_stub)

def callback(image_slice: np.ndarray) -> sv.Detections:
    result = yolo_pipeline(images=image_slice, iou_thres=0.6, conf_thres=0.001)
    return sv.Detections.from_deepsparse(result)

slicer = sv.InferenceSlicer(callback = callback)

detections = slicer(frame)
detections

[github-actions[bot]]

Hello there, thank you for opening an Issue ! 🙏🏻 The team was notified and they will get back to you asap.

[hardikdava]

@onuralpszr if you do not mind then I would like to tackle this one as I may have an idea.

[onuralpszr]

@hardikdava I already figure out

[[3.3687255859375003, -1.2968627929687502, 254.718115234375, 257.9044921875]] [[23.821523666381836, 317.9492492675781, 32.71510696411133, 319.9894714355469]] []

It was empty result we need skip or handle it. I also found couple more docs errors so I am going to open a PR If you don't mind :)

[onuralpszr]

@hardikdava I wanted to learn more and tackle more so I would like to finish this one

[hardikdava]

I am already about to finish. I found that the original issue is with sv.Detections.from_deepsparse() function. It has nothing to do with sv.InferenceSlicer()

[onuralpszr]

@hardikdava yeah same ?

[onuralpszr]

I found the issue in from_deepsparse as well

[onuralpszr]

def callback(image_slice: np.ndarray) -> sv.Detections:
    result = yolo_pipeline(images=image_slice, iou_thres=0.6, conf_thres=0.001)
    print(result.boxes[0])
    return sv.Detections.from_deepsparse(result)

I tested like this and see that empty array

[hardikdava]

Can you test with this callback?

def callback(image_slice: np.ndarray) -> sv.Detections:
    pipeline_outputs = yolo_pipeline(images=image_slice, iou_thres=0.6, conf_thres=0.001)
    if np.asarray(pipeline_outputs.boxes[0]).shape[0]>0:
        return sv.Detections.from_deepsparse(pipeline_outputs)
    else:
        return sv.Detections.empty()

[onuralpszr]

@hardikdava it works.

[hardikdava]

would you like to submit a PR or should I?

[hardikdava]

The bug is fixed in #348 PR. Closing as it is finished.

[SkalskiP]

@hardikdava and @onuralpszr, awesome work guys! 🔥 @mwitiderrick, did you have a chance to take a look at the current version in develop? Does it solve your issue?