Yolov8 on OAK-1 using depthai_sdk

[ptoupas]

Hi,

I am trying to deploy a custom yolov8 model to an OAK-1 camera using the depthai_sdk, however when I am executing the script I get the following error:

This is not a YOLO detection network! This configuration attempt will be ignored.

The code that I am running is the following:

with OakCamera() as oak:
    color = oak.create_camera('color')
    nn = oak.create_nn("yolov8s_cld.blob", color)
    with open(str(Path("yolov8s_cld.yaml").resolve())) as file:
        conf = json.load(file)
        nn.config_yolo_from_metadata(conf['nn_config']['NN_specific_metadata'])
    oak.visualize(nn.out.passthrough, fps=True)
    oak.start(blocking=True)

And the .yaml file content is as shown below:

{
    "nn_config":
    {
        "output_format" : "detection",
        "NN_family" : "YOLO",
        "input_size": "640x640",
        "NN_specific_metadata" :
        {
            "classes" : 2,
            "coordinates" : 4,
            "anchors" : [],
            "anchor_masks" : {},
            "iou_threshold" : 0.5,
            "confidence_threshold" : 0.5
        }
    },
    "mappings":
    {
        "labels":
        [
            "class-1",
            "class-2"
        ]
    }
}

I was wondering if the yolov8 models are supported in the depthai_sdk or if I have to try a different approach using the depthai API building the pipeline from the start?

[Erol444]

@ptoupas could you try:

nn = oak.create_nn("yolov8s_cld.blob", color, nn_type="yolo")

Otherwise you'd likely need to pass in json, which holds the metadata and model path, similar to jsons here: https://github.com/luxonis/depthai/blob/main/depthai_sdk/src/depthai_sdk/nn_models/yolo-v3-tiny-tf/config.json And instead of the model_name, you'd specify either blob, or xml + bin (openvino format, gets converted on-the-fly to blob).

[ptoupas]

@Erol444 Thanks for the quick reply!

By using the combination of adding the nn_type="yolo" argument when calling create_nn() and by using the .blob as produced by this tool https://tools.luxonis.com/ it seems to work now, thanks a lot!

I just wanted to give you some extra feedback. When I use the .blob model which I have converted from .onnx using this tool https://blobconverter.luxonis.com/ it seems to not working. I am not sure what is the issue, it might be something with the nms not being incorporated or something, but I get this error:

Mask is not defined for output layer with width '8400'. Define at pipeline build time using: 'setAnchorMasks' for 'side8400'.

Thank you for your assistance in quickly resolving my issue. Best!

[Erol444]

Hi @ptoupas , I believe that's because tools.luxonis.com will change the yolo architecture a bit, so also the anchor masks won't be the same as using blobconverter (which doesn't alter the architecture). Great to hear your issue was resolved! Thanks, Erik