search

khadas / edge2-npu

19 stars 6 forks source link

Custom YoloV5 Conversion #2

Closed Mitchelldscott closed 1 year ago

Mitchelldscott commented 1 year ago

I am using RKNN_toolkit2 to convert a YoloV5 model I am training. I have tried exporting the models with sdk 1.3.0 and am seeing segfaults.

eddge2-npu/C++/yolov5 custom model (without setting outputs in conversion)

post process config: box_conf_threshold = 0.50, nms_threshold = 0.60
Read data/img/bus.jpg ...
img width = 640, img height = 640
Loading mode...
sdk version: 1.3.0 (c193be371@2022-05-04T20:16:33) driver version: 0.7.2
model input num: 1, output num: 3
  index=0, name=images, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
  index=0, name=output, n_dims=4, dims=[1, 24, 80, 80], n_elems=153600, size=153600, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003823
  index=1, name=272, n_dims=4, dims=[1, 24, 40, 40], n_elems=38400, size=38400, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003779
  index=2, name=274, n_dims=4, dims=[1, 24, 20, 20], n_elems=9600, size=9600, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003544
model is NHWC input fmt
model input height=640, width=640, channel=3
once run use 18.226000 ms
loadLabelName ./data/coco_80_labels_list.txt
train @ (116 410 133 448) 0.726088
motorcycle @ (92 410 109 448) 0.726088
train @ (323 520 342 564) 0.726088
motorcycle @ (68 410 85 449) 0.726088
motorcycle @ (246 52 309 87) 0.726088
kite @ (2 188 53 223) 0.726088
train @ (50 381 117 430) 0.726088
cell phone @ (127 163 188 199) 0.726088
parking meter @ (10 414 29 445) 0.726088
train @ (268 52 303 87) 0.726088
surfboard @ (618 404 639 439) 0.726088
umbrella @ (594 542 618 592) 0.726088
bear @ (546 404 567 439) 0.726088
dog @ (474 403 495 437) 0.726088
fork @ (21 185 72 226) 0.726088
teddy bear @ (258 526 278 559) 0.726088
zebra @ (251 524 270 560) 0.726088
oven @ (124 139 191 176) 0.726088
parking meter @ (330 159 367 188) 0.726088
apple @ (410 389 429 452) 0.726088
sandwich @ (386 405 405 436) 0.726088
horse @ (372 402 389 438) 0.726088
train @ (204 52 271 87) 0.726088
potted plant @ (51 387 88 424) 0.726088
spoon @ (362 405 381 436) 0.726088
motorcycle @ (270 52 333 87) 0.726088
apple @ (348 402 365 439) 0.726088
elephant @ (314 404 333 436) 0.726088
frisbee @ (300 402 317 439) 0.726088
car @ (117 144 169 172) 0.726088
skateboard @ (26 393 94 418) 0.726088
oven @ (252 0 271 15) 0.726088
traffic light @ (188 52 239 87) 0.726088
cell phone @ (173 512 193 572) 0.726088
mouse @ (162 527 182 558) 0.726088
mouse @ (155 526 173 559) 0.726088
keyboard @ (276 402 293 439) 0.726088
oven @ (252 402 269 438) 0.726088
pizza @ (210 403 230 437) 0.726088
bus @ (186 406 206 440) 0.726088
tv @ (454 143 487 172) 0.726088
bicycle @ (126 509 144 570) 0.726088
cell phone @ (115 524 134 557) 0.726088
motorcycle @ (108 523 125 558) 0.726088
bottle @ (146 384 214 427) 0.726088
cow @ (180 405 197 441) 0.726088
hot dog @ (117 168 179 194) 0.726088
horse @ (156 405 173 441) 0.726088
potted plant @ (35 187 88 224) 0.726088
snowboard @ (378 522 398 564) 0.726088
tv @ (506 456 526 481) 0.726088
sheep @ (386 523 405 560) 0.726088
bicycle @ (98 405 117 435) 0.726088
hair drier @ (165 386 226 424) 0.726088
vase @ (195 387 234 424) 0.726088
mouse @ (27 527 46 558) 0.726088
parking meter @ (2 511 21 573) 0.726088
bicycle @ (74 405 93 436) 0.726088
dining table @ (110 159 175 204) 0.726088
motorcycle @ (60 403 76 437) 0.726088
backpack @ (26 405 45 436) 0.726088
cow @ (603 394 620 430) 0.726088
cow @ (586 394 606 429) 0.726088
person @ (572 516 606 553) 0.726088
Segmentation fault

eddge2-npu/C++/yolov5 custom model (with setting outputs in conversion)

post process config: box_conf_threshold = 0.50, nms_threshold = 0.60
Read data/img/bus.jpg ...
img width = 640, img height = 640
Loading mode...
sdk version: 1.3.0 (c193be371@2022-05-04T20:16:33) driver version: 0.7.2
model input num: 1, output num: 3
  index=0, name=images, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
  index=0, name=output, n_dims=4, dims=[1, 24, 80, 80], n_elems=153600, size=153600, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003823
  index=1, name=272, n_dims=4, dims=[1, 24, 40, 40], n_elems=38400, size=38400, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003779
  index=2, name=274, n_dims=4, dims=[1, 24, 20, 20], n_elems=9600, size=9600, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003544
model is NHWC input fmt
model input height=640, width=640, channel=3
once run use 18.245000 ms
loadLabelName ./data/coco_80_labels_list.txt
Segmentation fault
Mitchelldscott commented 1 year ago

Seems like an issue with the post-processing and difference in classes. I'll reopen this if I cant solve the error.

chakrish-edgeble commented 7 months ago

@Mitchelldscott if you don't mind can you share how to deploy for customised yolo model

Mitchelldscott commented 7 months ago

@chakrish-edgeble We were able to run a lighter model on the CPU so didn't keep trying. I'd checkout rknn_toolkit2, I definitely had issues following their process and understanding their compatibility.

chakrish-edgeble commented 7 months ago

@Mitchelldscott okay we are able to get inference for image and facing some issues with streaming and its quality