rockchip-linux / rknn-toolkit2

BSD 3-Clause "New" or "Revised" License
874 stars 154 forks source link

E RKNN: [17:19:34.670] failed to submit!, op id: 1, op name: Conv:/model.0/conv/Conv, flags: 0x1, task start: 0, task number: 30, run task counter: 0, int status: 0, please try updating to the latest version of the toolkit2 and runtime from: https://console.zbox.filez.com/l/I00fc3 (PWD: rknn) #320

Open livelove1987 opened 3 months ago

livelove1987 commented 3 months ago

Error apears only when trying to use first and second NPU core, when using core 0 everything works fine.

Getting such info before that error:

Message from syslogd@orangepi5plus at Jun 30 19:19:28 ...
 kernel:[  578.882333] Disabling IRQ #42

Message from syslogd@orangepi5plus at Jun 30 19:19:28 ...
 kernel:[  578.882333] Disabling IRQ #42

Also using such versions ( running on debian bookworm installed from orange pi software site 6.1.43-rockchip-rk3588 ) :

  RKNN: [17:19:24.679] RKNN Runtime Information, librknnrt version: 1.6.0 (9a7b5d24c@2023-12-13T17:31:11)
I RKNN: [17:19:24.690] RKNN Driver Information, version: 0.9.3
I RKNN: [17:19:24.720] RKNN Model Information, version: 6, toolkit version: 1.6.0+81f21f4d(compiler version: 1.6.0 (585b3edcf@2023-12-11T08:03:14)), target: RKNPU v2, target platform: rk3588, framework name: ONNX, framework layout: NCHW, model inference type: static_shape
Link164735 commented 3 months ago

I also encountered the same problem. Did you solve it?

livelove1987 commented 3 months ago

Yes, you need to update your NPU driver, it's hard to do it so you need to switch to different operating system or try to update it by yourself @Link164735

livelove1987 commented 3 months ago

https://github.com/Joshua-Riek/ubuntu-rockchip here is the ubuntu with updated NPU chip

Link164735 commented 3 months ago

https://github.com/Joshua-Riek/ubuntu-rockchip here is the ubuntu with updated NPU chip

Alright, I think I have a rough idea of what the issue is. Thank you very much

xiehousen commented 2 months ago

Have you solved this problem? I update the version of RKNN Server , Runtime and RKNN-Toolkit2 to 2.0.0b0 (35a6907d79@2024-03-24T02:34:11). But this error still occurs. Can you give me some advice?

Link164735 commented 2 months ago

Have you solved this problem? I update the version of RKNN Server , Runtime and RKNN-Toolkit2 to 2.0.0b0 (35a6907d79@2024-03-24T02:34:11). But this error still occurs. Can you give me some advice?

The reason for my issue was that the bit width of the model's output layer exceeded the maximum limit of the NPU, causing scheduling problems with the chip. It's important to note that the NPU's maximum bit width is 0x1fff, which is 8191. My original output layer was 2*9288. After modifying the dimensions of my output layer, the error disappeared. I hope this helps you.

Link164735 commented 2 months ago

Have you solved this problem? I update the version of RKNN Server , Runtime and RKNN-Toolkit2 to 2.0.0b0 (35a6907d79@2024-03-24T02:34:11). But this error still occurs. Can you give me some advice?

The reason for my issue was that the bit width of the model's output layer exceeded the maximum limit of the NPU, causing scheduling problems with the chip. It's important to note that the NPU's maximum bit width is 0x1fff, which is 8191. My original output layer was 2*9288. After modifying the dimensions of my output layer, the error disappeared. I hope this helps you.

image

xiehousen commented 2 months ago

Have you solved this problem? I update the version of RKNN Server , Runtime and RKNN-Toolkit2 to 2.0.0b0 (35a6907d79@2024-03-24T02:34:11). But this error still occurs. Can you give me some advice?

The reason for my issue was that the bit width of the model's output layer exceeded the maximum limit of the NPU, causing scheduling problems with the chip. It's important to note that the NPU's maximum bit width is 0x1fff, which is 8191. My original output layer was 2*9288. After modifying the dimensions of my output layer, the error disappeared. I hope this helps you.

image

Thank you. I encountered this situation by chance after running a few or dozens of frames, and it was the Resize function.

E RKNN: [01:05:13.527] failed to submit!, op id: 71, op name: Resize:/Resize, flags: 0x5, task start: 101, task number: 56, run task counter: 10, int status: 0, please try updating to the latest version of the toolkit2 and runtime from: https://console.zbox.filez.com/l/I00fc3 (PWD: rknn)