YoloV8 serverlesss support

[hardikdava]

My actions before raising this issue

• [x] Read/searched the docs
• [x] Searched past issues

Hello @cvat-maintainers :wave: , I can add support for YoloV8 object detection for automatic annotation. Please let me know if it helps cvat community for better and faster annotations then I would be happy to open a pull request. There is already a request from a user #5552

Future scope:

There is a lot of scope using ultralytics as they provide support for following models.

• YolloV5-YoloV8 instance segmentation
• YoloV8 pose estimation
• FastSAM
• RT-DETR
• Grounding-Dino (future)

[KTXKIKI]

Very much needed!!!!!! I hope it can be done. Thank you very much

[KTXKIKI]

@hardikdava
my brother Can you come up with other YOLOV8 segmentation, pose estimation, and trackers, as well as YOLOV8 and segment anything automatic object segmentation? I really need these. Thank you very much

[hardikdava]

@KTXKIKI I was thinking the same. It is already on my schedule.

[KTXKIKI]

@hardikdava import json import base64 from PIL import Image import io from ultralytics import YOLO from supervision.detection.utils import extract_yolov8_masks import supervision as sv

def init_context(context): context.logger.info("Init context... 0%")

model_path = "yolov8m-seg.pt"  #  YOLOV8模型放在nuclio目录下构建

model = YOLO(model_path)

# Read the DL model
context.user_data.model = model

context.logger.info("Init context...100%")

def handler(context, event): context.logger.info("Run yolo-v8 model") data = event.body buf = io.BytesIO(base64.b64decode(data["image"])) threshold = float(data.get("threshold", 0.35)) context.user_data.model.conf = threshold image = Image.open(buf)

yolo_results = context.user_data.model(image, conf=threshold)[0]
labels = yolo_results.names
detections = sv.Detections.from_yolov8(yolo_results)

detections = detections[detections.confidence > threshold]
boxes = detections.xyxy
conf = detections.confidence
class_ids = detections.class_id

results = []

if boxes.shape[0] > 0:

    for label, score, box in zip(class_ids, conf, boxes):

        xtl = int(box[0])
        ytl = int(box[1])
        xbr = int(box[2])
        ybr = int(box[3])

        mask = extract_yolov8_masks(yolov8_results)  # 调用 extract_yolov8_masks 函数获取多边形区域的掩码

        results.append({
                "confidence": str(score),
                "label": labels.get(label, "unknown"),
                "points": [xtl, ytl, xbr, ybr,  mask],
                "type": "rectangle",})

return context.Response(body=json.dumps(results), headers={},
                        content_type='application/json', status_code=200)

[KTXKIKI]

yolov8 segment I try to but have many problem

[hardikdava]

@KTXKIKI I got it working. But I have some issue with lots of polygon points. I am working on it. I will let you know once it is working.

[hardikdava]

@KTXKIKI checkout implementation from #6491. I successfully added suport for segmentation. I hope this will be helpful to you.

[KTXKIKI]

@hardikdava Thank you very much. I have also tried many ways, but there are always problems

[KTXKIKI]

YOLOV8 classification also requires

[rafaelgildin]

@KTXKIKI Is there any way of using roboflow as a support here? I know it's a problem exposing the data publicly or having to pay, but they are abble to take from COCO and export as YOLO dataset. Let me know if you found another strategy.