sipeed / MaixPy-v1

MicroPython for K210 RISC-V, let's play with edge AI easier
https://wiki.sipeed.com/maixpy
Other
1.68k stars 439 forks source link

[enhancement] Support for YOLOv3 #451

Open AIWintermuteAI opened 2 years ago

AIWintermuteAI commented 2 years ago

This adds support for running multi-branch YOLO v3 object detection network on MaixPy. Number of branches can be set to 1 or 2. Support for larger number of branches is possible, but will likely bring sub-optimal performance (low FPS). The YOLO v3 model was trained using aXeleRate framework. It has number of advantages if compared to YOLO v2 models, that I was using before:

Below is the best results I was able to achieve with YOLO v2 224x224 MobileNet 0.75 backbone IMG_20211210_181030 And here is (probably) not the best YOLO v3, trained for only 50 epochs 320x240 MobileNet 0.5 backbone IMG_20211210_181050

The out comparison for YOLOv2 and YOLOv3

"""
a = kpu.init_yolo2(task, 0.2, 0.1, 5, anchor)
>>> {"x":20, "y":30, "w":10, "h":21, "value":0.204172, "classid":1, "index":0, "objnum":6}
{"x":195, "y":38, "w":25, "h":33, "value":0.388860, "classid":0, "index":1, "objnum":6}
{"x":96, "y":65, "w":24, "h":52, "value":0.280810, "classid":1, "index":2, "objnum":6}
{"x":60, "y":95, "w":25, "h":33, "value":0.388278, "classid":0, "index":3, "objnum":6}
{"x":150, "y":125, "w":25, "h":51, "value":0.287573, "classid":0, "index":4, "objnum":6}
{"x":17, "y":174, "w":25, "h":33, "value":0.388692, "classid":0, "index":5, "objnum":6}
MicroPython v0.6.2-74-g8280e88f3-dirty on 2021-12-10; Sipeed_M1 with kendryte-k210
Type "help()" for more information.
"""
"""
a = kpu.init_yolo3(task, 0.5, 0.3, 3, 2, anchor)
>>> kpu_net __del__
YOLO v3
{"x":205, "y":135, "w":47, "h":68, "value":0.935626, "classid":0, "index":0, "objnum":11}
{"x":101, "y":8, "w":15, "h":17, "value":0.732297, "classid":0, "index":1, "objnum":11}
{"x":160, "y":14, "w":16, "h":24, "value":0.980218, "classid":0, "index":2, "objnum":11}
{"x":32, "y":28, "w":19, "h":24, "value":0.969574, "classid":1, "index":3, "objnum":11}
{"x":110, "y":36, "w":20, "h":22, "value":0.635944, "classid":1, "index":4, "objnum":11}
{"x":202, "y":46, "w":22, "h":30, "value":0.978948, "classid":0, "index":5, "objnum":11}
{"x":261, "y":51, "w":28, "h":33, "value":0.983265, "classid":0, "index":6, "objnum":11}
{"x":103, "y":68, "w":21, "h":32, "value":0.620853, "classid":0, "index":7, "objnum":11}
{"x":131, "y":78, "w":38, "h":50, "value":0.831876, "classid":1, "index":8, "objnum":11}
{"x":83, "y":96, "w":30, "h":40, "value":0.648528, "classid":1, "index":9, "objnum":11}
{"x":21, "y":182, "w":30, "h":32, "value":0.926121, "classid":0, "index":10, "objnum":11}
MicroPython v0.6.2-74-g8280e88f3-dirty on 2021-12-10; Sipeed_M1 with kendryte-k210
Type "help()" for more information.
>>>
"""

To maintain backwards compatibility, I have decided to preserve init_yolov2 and run_yolov2 functions and add init_yolov3 and run_yolov3 functions - this means code is somewhat clumsy, with nearly identical functions. Perhaps you have some suggestions or can make some edits yourself, @junhuanchen

I have compared the outputs for YOLOv2 inference on latest stable build of MaixPy and my fork, they are identical. Although more testing never hurts, if you'd like to do that.

"""
a = kpu.init_yolo2(task, 0.3, 0.2, 5, anchor)

kpu_net __del__
{"x":147, "y":31, "w":16, "h":52, "value":0.388692, "classid":0, "index":0, "objnum":3}
{"x":35, "y":86, "w":25, "h":51, "value":0.610327, "classid":0, "index":1, "objnum":3}
{"x":92, "y":89, "w":25, "h":52, "value":0.499636, "classid":0, "index":2, "objnum":3}
MicroPython v0.6.2-72-g22a8555b5 on 2021-11-01; Sipeed_M1 with kendryte-k210
Type "help()" for more information.
>>>
"""
"""
a = kpu.init_yolo2(task, 0.3, 0.2, 5, anchor)
>>> {"x":147, "y":31, "w":16, "h":52, "value":0.388692, "classid":0, "index":0, "objnum":3}
{"x":35, "y":86, "w":25, "h":51, "value":0.610327, "classid":0, "index":1, "objnum":3}
{"x":92, "y":89, "w":25, "h":52, "value":0.499636, "classid":0, "index":2, "objnum":3}
MicroPython v0.6.2-74-g8280e88f3-dirty on 2021-12-10; Sipeed_M1 with kendryte-k210
Type "help()" for more information.
>>>
"""
AIWintermuteAI commented 2 years ago

That would take care of these threads: https://github.com/sipeed/MaixPy/issues/272 https://github.com/sipeed/MaixPy/issues/178 https://github.com/sipeed/MaixPy/issues/173

junhuanchen commented 2 years ago

牛逼!!!!!!!牛逼!!!!!!!牛逼!!!!!!!牛逼!!!!!!!牛逼!!!!!!!

Abandon-ht commented 2 years ago

牛逼!!!!!!!牛逼!!!!!!!牛逼!!!!!!!牛逼!!!!!!!牛逼!!!!!!!

Abandon-ht commented 2 years ago

牛逼!!!!!!!牛逼!!!!!!!牛逼!!!!!!!牛逼!!!!!!!牛逼!!!!!!!

junhuanchen commented 2 years ago

Do not merge for now, pending attention

AIWintermuteAI commented 2 years ago

@junhuanchen any progress? :)

alexjones85 commented 2 years ago

I am trying to upload a YOLOv3 script via MaixPy IDE to the M.A.R.K vehicle with k210 module but it seems it is still only compatible with YOLOv2. Will the master branch of the repo be merged upgraded to YOLOv3 anytime soon?

junhuanchen commented 2 years ago

@junhuanchen any progress? :)

I'm watching, waiting for a good test.

TheMrViper commented 1 year ago

@junhuanchen waiting for good test, of feature that you dont have at all? maybe just merge it, some body will find bugs, and can write test, cause you just doing nothing here as devs. No updates, no fixes, no improvements. You are company, but you trying to get code for free, while you making money....

junhuanchen commented 1 year ago

@junhuanchen waiting for good test, of feature that you dont have at all? maybe just merge it, some body will find bugs, and can write test, cause you just doing nothing here as devs. No updates, no fixes, no improvements. You are company, but you trying to get code for free, while you making money....

Dear, I’m very sorry that this project has stopped updating from the perspective of the enterprise and will be handed over to community users to take care of and maintain in the future. If you need it or are responsible for this submission, you can fork and test it to give reasons for the need to be merged.

The reason why I can‘t merge is not that it can’t work, but that I can‘t be responsible for this submission, because it also needs corresponding training scripts and document tutorials behind it to communicate to community users that this function is feasible, including API descriptions and sample code.

Forgive me for not being able to merge. In fact, I knew it was available for a long time, but I didn’t have the conditions to deal with what needs to be completed after merging it.

You know, we can‘t make money from this open source project. We can only earn income by selling hardware. We have gradually migrated to Linux AI chips in China. You can find this fact on our wiki.sipeed.com.

Thank you very much for your attention. If necessary, I can add you to the warehouse administrator to handle these PRs. If possible, please inform me of your reasons and expectations. I very much welcome our open source users to take the initiative to maintain and update the open source project.

Love you, junhuanchen.

Abandon-ht commented 1 year ago

Hi! Very thanks u pull requests it.I want to know how to use aXeleRate train my custom yolov3 model by this Repositories config list only have yolov2 configs.How can i modified the config just like detector.json