object detection on video with deepsparse

[aymeric75]

Hello,

I could not find any example on how to use deepsparse for object detection on a video.

Here is the script that I use with a "normal" yolo8 model.

import cv2
from ultralytics import YOLO
import torch

model = YOLO('yolov8n.pt')

cap = cv2.VideoCapture('bowling.mov')

while cap.isOpened():

    success, frame = cap.read()

    if success:

        results = model(frame)
        annotated_frame = results[0].plot()
        cv2.imshow("YOLOv8n Inference", annotated_frame)

        if cv2.waitKey(1) & 0xFF == ord("q"):
            break

cap.release()
cv2.destroyAllWindows()

Any idea ?

Thanks !

[kXborg]

@aymeric75 I have the same query.

[mgoin]

Hi @aymeric75 and @kXborg

We have some exmaples of how to use YOLOv8 pipelines with DeepSparse here: https://github.com/neuralmagic/deepsparse/tree/main/src/deepsparse/yolov8. Particularly you might want to use the annotate.py script.

Here is an example command using that YOLOv8 script to annotate a video:

python deepsparse/src/deepsparse/yolov8/annotate.py --model_filepath zoo:cv/detection/yolov8-n/pytorch/ultralytics/coco/base-none --source apples.mp4

https://github.com/neuralmagic/deepsparse/assets/3195154/9bc23c73-3794-46db-95a2-a0f22e570db1

In the model_filepath you can see I used a SparseZoo model stub, which you can find for all the YOLOv8 COCO models here, optimized and base versions: https://sparsezoo.neuralmagic.com/?useCase=detection&datasets=coco&architectures=yolov8&ungrouped=true&sort=null

You can also replace python deepsparse/src/deepsparse/yolov8/annotate.py with deepsparse.yolov8.annotate as a CLI script.

We're working on more documentation for YOLOv8 so let me know if any more detail would help, thanks!

[jeanniefinks]

Hello @aymeric75 As it's been some time since we last communicated on this thread, I am going to go ahead and close out this inquiry. Should you have more questions, please reach out. Or a start a new issue on a new topic. Thank you! Jeannie / Neural Magic

[aymeric75]

I actually ended up using a simple SSD2 model, and a better hardware (coral edge tpu) that made the whole difference