search

airockchip / rknn-llm

Other
347 stars 29 forks source link

phi3 load error in rkllm1.0.1 #39

Closed eework closed 4 months ago

eework commented 4 months ago

我用rkllm1.0.1转换出phi3的模型，但是在板子上运行报错 运行命令: ./llm_demo ~/work/phi3-3_8b.rkllm 报错信息如下： rkllm init start rkllm-runtime version: 1.0.1

Query platform error!

之前在老版本的时候跑千问模型是ok的

eework commented 4 months ago

npu驱动是0.9.6，内核版本是5.10.110，硬件是rock5b 3588 rkllm-runtime是直接在板子进行编译的

(base) rock@rock-5b:/proc$ cat /sys/module/rknpu/version 0.9.6 (base) rock@rock-5b:/proc$ uname -a Linux rock-5b 5.10.110-50-rockchip-g5db04e2aa688 #rockchip SMP Thu May 2 02:52:04 PDT 2024 aarch64 GNU/Linux (base) rock@rock-5b:/proc$ (base) rock@rock-5b:/proc$ gcc -v Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/usr/lib/gcc/aarch64-linux-gnu/10/lto-wrapper Target: aarch64-linux-gnu Configured with: ../src/configure -v --with-pkgversion='Debian 10.2.1-6' --with-bugurl=file:///usr/share/doc/gcc-10/README.Bugs --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-10 --program-prefix=aarch64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-libquadmath --disable-libquadmath-support --enable-plugin --enable-default-pie --with-system-zlib --enable-libphobos-checking=release --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --enable-fix-cortex-a53-843419 --disable-werror --enable-checking=release --build=aarch64-linux-gnu --host=aarch64-linux-gnu --target=aarch64-linux-gnu --with-build-config=bootstrap-lto-lean --enable-link-mutex Thread model: posix Supported LTO compression algorithms: zlib zstd gcc version 10.2.1 20210110 (Debian 10.2.1-6) (base) rock@rock-5b:/proc$

lzz773751548 commented 4 months ago

您可以尝试在板端 cat /proc/device-tree/compatible看看是否能获得芯片信息

eework commented 4 months ago

您可以尝试在板端 cat /proc/device-tree/compatible看看是否能获得芯片信息 可以获取到的 (base) rock@rock-5b:~$ cat /proc/device-tree/compatible radxa,rock-5brockchip,rk3588(base) rock@rock-5b:~$

eework commented 4 months ago

image

lzz773751548 commented 4 months ago

您可以尝试给可执行文件root权限后再执行

eework commented 4 months ago

root执行结果也是一样的 image

eework commented 4 months ago

请问，软件逻辑上，什么情况下会出现“Query platform error!”？ 还没执行到加载模型那一步

eework commented 4 months ago

调试发现rkllm_init(&llmHandle, param, callback);的调用会出现“Query platform error!”？，但里面实现看不到，不知道为什么

eework commented 4 months ago

更新1.0.2已经解决