sipeed / MaixPy-v1

MicroPython for K210 RISC-V, let's play with edge AI easier
https://wiki.sipeed.com/maixpy
Other
1.68k stars 438 forks source link

YOLO V3 support, KPU Failed to load kmodel with multi output layers #178

Open zyayoung opened 5 years ago

zyayoung commented 5 years ago

When i want to try yolov3, I don't think it is too hard to modify reigon layers. However, I find i can't even load my kmodel with two output layers.

kmodel from nncase v0.1.0-rc5

same kmodel works fine on my c program

import sensor
import image
import lcd
import KPU as kpu

lcd.init()
sensor.reset()
sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.QVGA)
sensor.run(1)
task = kpu.load("/sd/yolo.kmodel") #使用kfpkg将 kmodel 与 maixpy 固件打包下载到 flash
while(True):
    img = sensor.snapshot()
    a = lcd.display(img)
Traceback (most recent call last):
  File "<stdin>", line 11, in <module>
ValueError: $>>> init i2c2

Model (works perfectly on my c program)

0: InputLayer -> 1x3x224x320
1: K210Conv2d 1x3x224x320 -> 1x4x224x320
2: K210Conv2d 1x4x224x320 -> 1x4x224x320
3: K210Conv2d 1x4x224x320 -> 1x8x224x320
4: K210Conv2d 1x8x224x320 -> 1x8x224x320
5: K210Conv2d 1x8x224x320 -> 1x16x112x160
6: K210Conv2d 1x16x112x160 -> 1x16x112x160
7: K210Conv2d 1x16x112x160 -> 1x16x112x160
8: K210Conv2d 1x16x112x160 -> 1x16x112x160
9: K210Conv2d 1x16x112x160 -> 1x32x56x80
10: K210Conv2d 1x32x56x80 -> 1x32x56x80
11: K210Conv2d 1x32x56x80 -> 1x32x56x80
12: K210Conv2d 1x32x56x80 -> 1x32x56x80
13: K210Conv2d 1x32x56x80 -> 1x64x28x40
14: K210Conv2d 1x64x28x40 -> 1x64x28x40
15: K210Conv2d 1x64x28x40 -> 1x64x28x40
16: K210Conv2d 1x64x28x40 -> 1x64x28x40
17: K210Conv2d 1x64x28x40 -> 1x64x28x40
18: K210Conv2d 1x64x28x40 -> 1x64x28x40
19: K210Conv2d 1x64x28x40 -> 1x64x28x40
20: K210Conv2d 1x64x28x40 -> 1x64x28x40
21: K210Conv2d 1x64x28x40 -> 1x64x28x40
22: K210Conv2d 1x64x28x40 -> 1x64x28x40
23: K210Conv2d 1x64x28x40 -> 1x64x28x40
24: K210Conv2d 1x64x28x40 -> 1x64x28x40
25: K210Conv2d 1x64x28x40 -> 1x112x14x20
26: K210Conv2d 1x112x14x20 -> 1x112x14x20
27: K210Conv2d 1x112x14x20 -> 1x112x14x20
28: K210Conv2d 1x112x14x20 -> 1x128x14x20
29: K210Conv2d 1x128x14x20 -> 1x24x14x20
30: Dequantize 1x24x14x20 -> 1x24x14x20
31: OutputLayer 1x24x14x20
32: K210Conv2d 1x112x14x20 -> 1x128x14x20
33: QuantizedResizeNearestNeighbor 1x128x14x20 -> 1x128x28x40
34: Requantize 1x128x28x40 -> 1x128x28x40
35: Requantize 1x64x28x40 -> 1x64x28x40
36: QuantizedConcatenation 1x128x28x40 -> 1x192x28x40
37: K210Upload 1x192x28x40 -> 1x192x28x40
38: K210Conv2d 1x192x28x40 -> 1x128x28x40
39: K210Conv2d 1x128x28x40 -> 1x24x28x40
40: Dequantize 1x24x28x40 -> 1x24x28x40
41: OutputLayer 1x24x28x40
KPU memory usage: 2097152 B
Main memory usage: 642880 B
zyayoung commented 4 years ago

I implemented yolov3 with kendryte SDK, and it does much better on detecting objects of different scales, so i hope it can be implemented on Maixpy.

I think this issue #178 might be the only issue that prevent me from using yolov3 in Maixpy.

Here is a v3 demo, you can see how good it can be. If we can use yolov3 on Maxipy, it will be much more convenient to use these detection results to do some analysis.

Also, I'm willing to help :-)

kirii7 commented 4 years ago

Hi @zyayoung , i get the same error while trying to load a yolo v2 model. did u find a workaround to prevent this error? Also, could you let me know how you implemented the yolo v3 in c language? Thanks in advance!