search

rockchip-linux / rknn-toolkit2

BSD 3-Clause "New" or "Revised" License
828 stars 151 forks source link

Run Multiple Models at the same time on RK3588 #297

Closed zaferarican closed 3 months ago

zaferarican commented 3 months ago

Is it possible to run multiple models on RK3588 by distributing them to different cores by using for example rknn_set_core_mask?

When I try to test core_mask using the rank_benchmark example, I get

E RKNN: [17:26:07.646] failed to submit!, op id: 1, op name: Conv:Conv_0, flags: 0x5, task start: 618, task number: 21, run task counter: 0, int status: 0, please try updating to the latest version of the toolkit2 and runtime from: https://console.zbox.filez.com/l/I00fc3 (PWD: rknn) rknn run error -1

Is it even possible to use multiple cores in RK3588?

I am running on Orange Pi 5 Plus with 16 GB Ram.

zaferarican commented 3 months ago

Never mind. I figured out that it was due to the kernel 6.1.43. It worked on 5.10.

TakeshiGianlee commented 3 months ago

what version of rknntoolkit2 do you use? 2.0.0?

zaferarican commented 3 months ago

It works both on 1.6.0 and 2.0.0. For me the key point was using the 5.10 kernel for Ubuntu, not the 6.1.43 version.

core_mask is used to select which core to use. RK3588 has 3 NPU cores so you can run 3 models at the same time.

TakeshiGianlee commented 3 months ago

thanks for your reply.I also have a issue.When i run my code with kernel 5.10,it reminds me that my Rknn driver version is too low(0.8.2),does it matter?

TakeshiGianlee commented 3 months ago

thanks for your reply.I also have a issue.When i run my code with kernel 5.10,it reminds me that my Rknn driver version is too low(0.8.2),does it matter?

It doesnt matter at all.

zaferarican commented 3 months ago

Mine is also 0.8.2. It seems ok. I did not have a problem. I did not perform comprehensive tests though.