yolox -> onnx -> ncnn: Final converted ncnn network not working

[realElMartino]

Hi,

1) I train a custom yolox-s network 2) It works fine when testing with .\tools\demo.py 3) I export the onnx model using simplify 4) Convert to ncnn using onnx2ncnn 5) I follow all instructions to replace the head of the .param file

original: 7767517 235 268 Input images 0 1 images Split splitncnn_input0 1 4 images images_splitncnn_0 images_splitncnn_1 images_splitncnn_2 images_splitncnn_3 Crop Slice_4 1 1 images_splitncnn_3 467 -23309=1,0 -23310=1,2147483647 -23311=1,1 Crop Slice_9 1 1 467 472 -23309=1,0 -23310=1,2147483647 -23311=1,2 Crop Slice_14 1 1 images_splitncnn_2 477 -23309=1,0 -23310=1,2147483647 -23311=1,1 Crop Slice_19 1 1 477 482 -23309=1,1 -23310=1,2147483647 -23311=1,2 Crop Slice_24 1 1 images_splitncnn_1 487 -23309=1,1 -23310=1,2147483647 -23311=1,1 Crop Slice_29 1 1 487 492 -23309=1,0 -23310=1,2147483647 -23311=1,2 Crop Slice_34 1 1 images_splitncnn_0 497 -23309=1,1 -23310=1,2147483647 -23311=1,1 Crop Slice_39 1 1 497 502 -23309=1,1 -23310=1,2147483647 -23311=1,2 Concat Concat_40 4 1 472 492 482 502 503 0=0 Convolution Conv_41 1 1 503 877 0=32 1=3 11=3 2=1 12=1 3=1 13=1 4=1 14=1 15=1 16=1 5=1 6=3456

Modified: 7767517 226 268 Input images 0 1 images Input focus 1 1 images 503 Convolution Conv_41 1 1 503 877 0=32 1=3 11=3 2=1 12=1 3=1 13=1 4=1 14=1 15=1 16=1 5=1 6=3456

6) Then using ncnnoptimize with fp16 switch 65536 7) Replace second dummy input with YoloV5Focus 7767517 220 250 Input images 0 1 images YoloV5Focus focus 1 1 images 503 Convolution Conv_41 1 1 503 877 0=32 1=3 4=1 5=1 6=3456 ...

When doing this with a yolox-tiny network I end up with the exact same network as a provided working demo except the number of classes in the output which is 191 in my case.

Now everything looks fine but the network does not perform at all - like not working at all. It results in one class being permanently being detected. For training of the final network I use 640x640 images and this is used in training and also set in the app. I also tested tiny networks with 416x416 but always the same result.

The environment I train and test is on Windows using onnx 1.10.1, onnxruntime 1.9.0, onnx-simplifier 0.3.6. Using yolox 0.1.0 and latest ncnn. I have tested almost everything over weeks now and just can't find what the issue could be. I also looked at other posts where they suggested to use onnx version 1.8.1, onnxruntime 1.8.0 and onnx-simplifier 0.3.5 - but also having this set in an environment it is the same result as well as doing the conversion on a linux box. Followed instructions here https://zhuanlan.zhihu.com/p/391788686 and others but always end up with the same non-performing network in ncnn.

Please let me know what I'm missing.

[jackchunliu]

I have the same problem! Have you solved? I build yolox.cpp on ubuntu, it can detect. But on Windows 10 ,it can't detect anything!

[realElMartino]

Sorry but I gave up on it and used YOLOv5 instead. I needed this for an experiment and could no longer waste time on it and no one replied here either. YOLOv5 just worked when following the process. For me accuracy was more important than speed - anyway, so it did not matter but I would have liked to compare the anchorless approach. Maybe after some more development and tidy up of code this project will work better for testing.

[HyperScypion]

I've also tried to convert my yolox models with the same instruction and also using https://convertmodel.com/ I've got the same issue. I've compared param files as well as binaries and only binaries are different.

[HyperScypion]

Hey I found some kind of the solution that worked for me. Use this version of the ncnn library https://github.com/Tencent/ncnn/releases/tag/20210525 and convert model base on it. The first issue that state this problem is from April 2021 so maybe there is a bug or some kind of problems with libraries which are from April or later.

[superbayes]

Sorry but I gave up on it and used YOLOv5 instead. I needed this for an experiment and could no longer waste time on it and no one replied here either. YOLOv5 just worked when following the process. For me accuracy was more important than speed - anyway, so it did not matter but I would have liked to compare the anchorless approach. Maybe after some more development and tidy up of code this project will work better for testing.

Use yolov5 and ncnn? Did you deploy successfully?

[realElMartino]

Sorry but I gave up on it and used YOLOv5 instead. I needed this for an experiment and could no longer waste time on it and no one replied here either. YOLOv5 just worked when following the process. For me accuracy was more important than speed - anyway, so it did not matter but I would have liked to compare the anchorless approach. Maybe after some more development and tidy up of code this project will work better for testing.

Use yolov5 and ncnn? Did you deploy successfully?

Yes - that worked fine. Just follow the same procedure. When exporting the ONNX model from YOLOv5 you'll have to use --train to get the correct model. e.g. export.py --weights yourweightfile.pt --img 640 640 --batch 1 --simplify --opset 11 --train --include onnx Then do simplify: python -m onnxsim yourmodel.onnx yourmodel_sim.onnx then onnx2ncnn ... then fix your header whether you use the custom layer or not then do ncnnoptimize ...then it should be fine to deploy on the mobile if you set the app parameters right.

[superbayes]

Sorry but I gave up on it and used YOLOv5 instead. I needed this for an experiment and could no longer waste time on it and no one replied here either. YOLOv5 just worked when following the process. For me accuracy was more important than speed - anyway, so it did not matter but I would have liked to compare the anchorless approach. Maybe after some more development and tidy up of code this project will work better for testing.

Use yolov5 and ncnn? Did you deploy successfully?

Yes - that worked fine. Just follow the same procedure. When exporting the ONNX model from YOLOv5 you'll have to use --train to get the correct model. e.g. export.py --weights yourweightfile.pt --img 640 640 --batch 1 --simplify --opset 11 --train --include onnx Then do simplify: python -m onnxsim yourmodel.onnx yourmodel_sim.onnx then onnx2ncnn ... then fix your header whether you use the custom layer or not then do ncnnoptimize ...then it should be fine to deploy on the mobile if you set the app parameters right.

Are you using yolov5 release 6?

[realElMartino]

Are you using yolov5 release 6?

I remember having had an issue after an upgrade as the model for r6 changed and I think I continued my project sticking with the previous release. I wanted to check accuracies with the 1280 resolution but never did. However, should be no problem to use it just to change parameters in the runtime then.

[superbayes]

Are you using yolov5 release 6?

I remember having had an issue after an upgrade as the model for r6 changed and I think I continued my project sticking with the previous release. I wanted to check accuracies with the 1280 resolution but never did. However, should be no problem to use it just to change parameters in the runtime then.

thanks. I use yolov5 r6(yolov5s.pt),deploy with ncnn sucessfully. but,When I use yolov5 r6(yolov5n.pt),deploy with ncnn failed. The ncnn model predict nothing.I don't know what's wrong. and following figure is yolov5s ncnn model predict result

[realElMartino]

Are you using yolov5 release 6?

I remember having had an issue after an upgrade as the model for r6 changed and I think I continued my project sticking with the previous release. I wanted to check accuracies with the 1280 resolution but never did. However, should be no problem to use it just to change parameters in the runtime then.

thanks. I use yolov5 r6(yolov5s.pt),deploy with ncnn sucessfully. but,When I use yolov5 r6(yolov5n.pt),deploy with ncnn failed. The ncnn model predict nothing.I don't know what's wrong. and following figure is yolov5s ncnn model predict result

What are you doing in your app config - have you set the output nodes correctly? Not sure about your experience level so just guessing. The nodes will be different and you'll have to change that in case you just use a ready app that was built for a small model.

[nihui]

针对onnx模型转换的各种问题，推荐使用最新的pnnx工具转换到ncnn In view of various problems in onnx model conversion, it is recommended to use the latest pnnx tool to convert your model to ncnn

pip install pnnx
pnnx model.onnx inputshape=[1,3,224,224]

详细参考文档 Detailed reference documentation https://github.com/pnnx/pnnx https://github.com/Tencent/ncnn/wiki/use-ncnn-with-pytorch-or-onnx#how-to-use-pnnx