the model version is too old?

[thanhtantran]

I try to install it on Orange Pi 5 Plus and get this error

rkllm init start
rkllm-runtime version: 1.0.1
Warning: The model version is too old, please use the latest toolkit to reconvert the model!
The model target_platform does not match!
RKLLM: failed to load model '/home/orangepi/qwen-chat-1_8B.rkllm'!!!
RKLLM init failed!

I have downloaded the model from this link https://huggingface.co/Pelochus/qwen-1_8B-rk3588

[av1d]

Thank you for bringing this to my attention. This is because that model was converted for RKLLM 1.0 and the current server version is for 1.0.1. I just pushed the legacy version to the repo as the file server_1_0.cpp, this will run that model.

[thanhtantran]

while compiling the server_1_0.cpp, i got error from 131 | param.modelpath = rkllm_model.c_str(); this error i have fixed with change modelpath to model_path

however, this error i don't know where to fix it

server_1_0.cpp:138:52: error: invalid conversion from ‘void (*)(const char*, void*, LLMCallState)’ to ‘LLMResultCallback’ {aka ‘void (*)(RKLLMResult*, void*, LLMCallState)’} [-fpermissive]
  138 |     int initResult = rkllm_init(&llmHandle, param, callback);
      |                                                    ^~~~~~~~
      |                                                    |
      |                                                    void (*)(const char*, void*, LLMCallState)
In file included from server_1_0.cpp:13:

[av1d]

Are you using ezrknpu or RKLLM / RKNN, and did you pull the latest version or an older version? Just wondering because the latest version of either is 1.0.1 and the rkllm.h shared library changed since 1.0, so it is incompatible with the older server version (server_1_0.cpp). If you pulled the newer one, can you try with this instead?

[thanhtantran]

I am using ezrknpu , and already pull the old brand, install it but still get the error as above.

Is there any way that i can check which RKLLM version i am using now?

[thanhtantran]

i found how to check which RKLLM version is running by typing command rklllm model

i still using the version 1.0.1

now don't know how to uninstall the version 1.0.1 and reinstall the version 1.0.0

[av1d]

You can find the location of the binary by typing which rkllm. It's likely located at /usr/bin/rkllm. If you wanted to remove it you can rename it or delete it, but this binary shouldn't affect compiling the server, so long as the server is compiled from within the correct version of the toolkits. I would be more concerned that the rkllm.h (probably located at /usr/local/include/rkllm.h) file doesn't match the correct version. Can you make a new directory then put this script into it? Name it something like environment.sh then execute it by typing bash environment.sh. It will save your environment information into a text file, which may help debug.

[dezhishen]

You can find the location of the binary by typing which rkllm. It's likely located at /usr/bin/rkllm. If you wanted to remove it you can rename it or delete it, but this binary shouldn't affect compiling the server, so long as the server is compiled from within the correct version of the toolkits. I would be more concerned that the rkllm.h (probably located at /usr/local/include/rkllm.h) file doesn't match the correct version. Can you make a new directory then put this script into it? Name it something like environment.sh then execute it by typing bash environment.sh. It will save your environment information into a text file, which may help debug.

use rk3588s

================================
RAM: 3921
================================
Disk usage: 465G total, 30G used, 431G available, 7% used%
================================
Distributor ID: Debian
Description:    Armbian 24.5.1 bookworm
Release:        12
Codename:       bookworm
================================
Kernel version: 5.10.160-legacy-rk35xx
================================
RKLLM.h version: RKLLM 1.0.1
================================
Python version: Python 3.11.2
================================
dmesg output:
[    5.187024] rk_gmac-dwmac fe1c0000.ethernet: clock input or output? (output).
[    5.493307] input: rk805 pwrkey as /devices/platform/feb20000.spi/spi_master/spi2/spi2.0/rk805-pwrkey.6.auto/input/input0
[    5.514809] vdd_npu_s0: supplied by vcc5v0_sys
[    5.518888] vdd_npu_s0: 550 <--> 950 mV at 800 mV, enabled
[    5.565130] rkisp_hw fdcb0000.rkisp: max input:0x0@0fps
[    5.565930] rkisp_hw fdcc0000.rkisp: max input:0x0@0fps
[    5.665242] input: rockchip-dp0 rockchip-dp0 as /devices/platform/dp0-sound/sound/card0/input1
[    5.667059] input: rockchip-hdmi0 rockchip-hdmi0 as /devices/platform/hdmi0-sound/sound/card1/input2
[    5.668212] input: headset-keys as /devices/platform/es8388-sound/input/input3
[    5.733425] input: rockchip-es8388 Headset as /devices/platform/es8388-sound/sound/card2/input4
[    5.786075] input: adc-keys as /devices/platform/adc-keys/input/input5
[    5.790229] rockchip-pm-domain fd8d8000.power-management:power-controller: Looking up npu-supply from device tree
[    5.790265] rockchip-pm-domain fd8d8000.power-management:power-controller: Looking up npu-supply property in node /power-management@fd8d8000/power-controller failed
[    5.792608] RKNPU fdab0000.npu: Adding to iommu group 0
[    5.792828] RKNPU fdab0000.npu: RKNPU: rknpu iommu is enabled, using iommu mode
[    5.792972] RKNPU fdab0000.npu: Looking up rknpu-supply from device tree
[    5.793814] RKNPU fdab0000.npu: Looking up mem-supply from device tree
[    5.794531] RKNPU fdab0000.npu: can't request region for resource [mem 0xfdab0000-0xfdabffff]
[    5.794565] RKNPU fdab0000.npu: can't request region for resource [mem 0xfdac0000-0xfdacffff]
[    5.794588] RKNPU fdab0000.npu: can't request region for resource [mem 0xfdad0000-0xfdadffff]
[    5.795292] [drm] Initialized rknpu 0.9.2 20231018 for fdab0000.npu on minor 1
[    5.795716] rockchip-pm-domain fd8d8000.power-management:power-controller: Looking up nputop-supply from device tree
[    5.795739] rockchip-pm-domain fd8d8000.power-management:power-controller: Looking up nputop-supply property in node /power-management@fd8d8000/power-controller failed
[    5.795810] rockchip-pm-domain fd8d8000.power-management:power-controller: Looking up npu1-supply from device tree
[    5.795830] rockchip-pm-domain fd8d8000.power-management:power-controller: Looking up npu1-supply property in node /power-management@fd8d8000/power-controller failed
[    5.795887] rockchip-pm-domain fd8d8000.power-management:power-controller: Looking up npu2-supply from device tree
[    5.795906] rockchip-pm-domain fd8d8000.power-management:power-controller: Looking up npu2-supply property in node /power-management@fd8d8000/power-controller failed
[    5.796026] RKNPU fdab0000.npu: Looking up rknpu-supply from device tree
[    5.796043] vdd_npu_s0: could not add device link fdab0000.npu: -EEXIST
[    5.796046] vdd_npu_s0: Failed to create debugfs directory
[    5.796604] RKNPU fdab0000.npu: Looking up mem-supply from device tree
[    5.796622] vdd_npu_s0: could not add device link fdab0000.npu: -EEXIST
[    5.796626] vdd_npu_s0: Failed to create debugfs directory
[    5.797269] RKNPU fdab0000.npu: Looking up rknpu-supply from device tree
[    5.797286] vdd_npu_s0: could not add device link fdab0000.npu: -EEXIST
[    5.797289] vdd_npu_s0: Failed to create debugfs directory
[    5.799200] RKNPU fdab0000.npu: RKNPU: bin=0
[    5.799375] RKNPU fdab0000.npu: leakage=8
[    5.799398] RKNPU fdab0000.npu: Looking up rknpu-supply from device tree
[    5.799415] debugfs: Directory 'fdab0000.npu-rknpu' with parent 'vdd_npu_s0' already present!
[    5.807607] RKNPU fdab0000.npu: pvtm=873
[    5.813542] RKNPU fdab0000.npu: pvtm-volt-sel=3
[    5.815213] RKNPU fdab0000.npu: avs=0
[    5.815585] RKNPU fdab0000.npu: l=10000 h=85000 hyst=5000 l_limit=0 h_limit=800000000 h_table=0
[    5.827866] RKNPU fdab0000.npu: failed to find power_model node
[    5.827880] RKNPU fdab0000.npu: RKNPU: failed to initialize power model
[    5.827890] RKNPU fdab0000.npu: RKNPU: failed to get dynamic-coefficient
[    6.138863] input: USB Keyboard USB Keyboard as /devices/platform/fc840000.usb/usb3/3-1/3-1:1.0/0003:1A81:203E.0001/input/input6
[    6.139105] input: USB Keyboard USB Keyboard Consumer Control as /devices/platform/fc840000.usb/usb3/3-1/3-1:1.0/0003:1A81:203E.0001/input/input7
[    6.197976] hid-generic 0003:1A81:203E.0001: input,hiddev96,hidraw0: USB HID v1.10 Device [USB Keyboard USB Keyboard] on usb-fc840000.usb-1/input0
[    6.205575] input: USB Keyboard USB Keyboard as /devices/platform/fc840000.usb/usb3/3-1/3-1:1.1/0003:1A81:203E.0002/input/input8
[    6.261630] hid-generic 0003:1A81:203E.0002: input,hidraw1: USB HID v1.10 Keyboard [USB Keyboard USB Keyboard] on usb-fc840000.usb-1/input1
[    6.267663] hid-generic 0003:1A81:203E.0003: hiddev97,hidraw2: USB HID v1.10 Device [USB Keyboard USB Keyboard] on usb-fc840000.usb-1/input2
[  351.360041] RKNPU fdab0000.npu: RKNPU: set rknpu freq: 1000000000, volt: 812500