Qengineering / YoloV8-ncnn-Raspberry-Pi-4

YoloV8 for a bare Raspberry Pi 4 or 5
https://qengineering.eu/deep-learning-examples-on-raspberry-32-64-os.html
BSD 3-Clause "New" or "Revised" License
107 stars 7 forks source link

how to import model Yolov8 for ncnn #2

Open pastukhov-aleksandr opened 1 year ago

pastukhov-aleksandr commented 1 year ago

Good afternoon. how to import a trained model for ncnn correctly?

I tried:

!yolo mode=export model={HOME}/runs/detect/train/weights/best.pt format=onnx
!python -m onnxsim {HOME}/runs/detect/train/weights/best.onnx {HOME}/runs/detect/train/weights/best-sim.onnx

and onnx2ncnn best-sim.onnx best.param best.bin

I get an error:

find_blob_index_by_name output failed
Try
    ex.extract("output0", out0);

best.param:

7767517
210 255
Input            images                   0 1 images
MemoryData       /model.22/Constant_3_output_0 0 1 /model.22/Constant_3_output_0 0=2
MemoryData       /model.22/Constant_4_output_0 0 1 /model.22/Constant_4_output_0 0=8400 1=2
Split            splitncnn_0              1 2 /model.22/Constant_4_output_0 /model.22/Constant_4_output_0_splitncnn_0 /model.22/Constant_4_output_0_splitncnn_1
MemoryData       /model.22/Constant_7_output_0 0 1 /model.22/Constant_7_output_0 0=8400
MemoryData       onnx::Split_140          0 1 onnx::Split_140 0=2
MemoryData       onnx::Split_160          0 1 onnx::Split_160 0=2
Split            splitncnn_1              1 2 onnx::Split_160 onnx::Split_160_splitncnn_0 onnx::Split_160_splitncnn_1
MemoryData       onnx::Split_187          0 1 onnx::Split_187 0=2
Split            splitncnn_2              1 3 onnx::Split_187 onnx::Split_187_splitncnn_0 onnx::Split_187_splitncnn_1 onnx::Split_187_splitncnn_2
MemoryData       onnx::Split_214          0 1 onnx::Split_214 0=2
Qengineering commented 1 year ago

@pastukhov-aleksandr,

Porting from .pt to ncnn can be a nightmare. In your case ncnn can't find a proper output layer. It suggests you use the layer "output0". I hope it works. Most of the time more things need to be adapted before your .pt model works in the ncnn-framework. see also: https://github.com/FeiGeChuanShu/ncnn-android-yolov8/issues/1

If you don't succeed, you can send me the bin and param file and I see what I can do. No promises, I'm very busy right now. However, at some point I will need to walk the same path as you are about to do.

If you succeed, please let me know and we can put some suggestions and hits in the readme.

pastukhov-aleksandr commented 1 year ago

@Qengineering my model file https://drive.google.com/drive/folders/1HZLjMDFv6JSaL5EXuPCFEFpLHniJ2QCh?usp=share_link

Qengineering commented 1 year ago

Got them! (Remember, no promises.)

apanand14 commented 1 year ago

Hello @Qengineering @pastukhov-aleksandr Are you able to solve your issue? I'm also getting the exact same .param file after converting my model from ONNX.

pastukhov-aleksandr commented 1 year ago

I am waiting for the publication of the solution https://github.com/Tencent/ncnn/issues/4195

yrik commented 1 year ago

Hi @Qengineering thank you for the awesome work.

I've tried to export a custom model and use it with your code. But it was not successful. I've described all the steps here https://github.com/FeiGeChuanShu/ncnn-android-yolov8/issues/8#issuecomment-1610283859

My problem is that it finds 1655 objects in the picture where there is only one relevant object.

I made a public google colab notebook so it's easier to see all the steps I made https://colab.research.google.com/drive/1rZf3f7koBCyGOfZKiOutPtl0El6w6TfR?usp=sharing

Also I've uploaded the .params file https://github.com/FeiGeChuanShu/ncnn-android-yolov8/files/11887677/old-best-sim-opt-fp16.param.txt

Would appreciate any help or hints on how to solve the issue.

Btw, I've noticed that on ncnn-android-yolov8 they also introduce Crop layer. Could it be the issue?

Qengineering commented 1 year ago

@yrik,

I'm sorry. At the moment I'm busy with a lot of projects. To be honest, it is hard to find time to investigate the issue. Knowing FeiGeChuanShu, you're in good hands. If there is someone how can solve your issue, it's him.

dceluis commented 1 year ago

Have a look @ https://github.com/ultralytics/ultralytics/issues/3412#issuecomment-1625406143

Qengineering commented 1 year ago

That's the news we all waiting for.