search

AndreyGermanov / yolov8_onnx_rust

YOLOv8 inference using Rust
GNU General Public License v3.0
51 stars 7 forks source link

[Question] Segmentation #3

Closed NBarile closed 8 months ago

NBarile commented 8 months ago

Hi !

I just come to ask you if it's possible to make segmentation with your module ?

Thanks !

AndreyGermanov commented 8 months ago

Hi, thanks for your interest!

Not exactly, but this module can be used as a base for segmentation. It's required to load the YOLOv8 model for segmentation and rewrite the "process_output" part.

I explained how to do this with ONNX in this article: https://dev.to/andreygermanov/how-to-implement-instance-segmentation-using-yolov8-neural-network-3if9. But it has only Python example for now.

You can use this module and this article to rewrite that Python code on Rust. Perhaps I will find time to do this later too.

Honestly speaking, I found that the YOLOv8 model is not the best choice for segmentation. The quality of the masks that it produces is far from the best, because it can produce only 160x160 masks that should be scaled to original size.

YOLOv8 is great for detection, but for segmentation the Segment Anything model produces a way better high resolution masks. I created a Rust sample for this, if interested: https://github.com/AndreyGermanov/sam_onnx_rust. However, it works slower that YOLO, so it's up to you what to choose depending on tasks.

AndreyGermanov commented 8 months ago

Hi, I have created a similar web application, that can be used both for object detection and segmentation. See it here: https://github.com/AndreyGermanov/yolov8_onnx_rust_segmentation.

This is the same code as in this repository, but modified to return both bounding boxes and segmentation masks.

Closing this issue ...