search

Pelochus / ezrknn-llm

Easier usage of LLMs in Rockchip's NPU on SBCs like Orange Pi 5 and Radxa Rock 5 series
Other
32 stars 3 forks source link

do you know how to set up RKLLM gradio server? #7

Open kiron111 opened 1 month ago

kiron111 commented 1 month ago

my sbc model: orange pi 5 plus my system: armbian 24.5.0 (Armbian-Pelochus-24.5.0-trunk-orangepi5plus_jammy_standard_vendor_6.1.43.img)

The official PDF guide suggests utilizing the adb shell command to effortlessly assemble RKLLM-Server-Gradio. However, upon connecting the Orangepi via a Type-C cable, the device fails to establish a connection with ,my PC (I tried on Windows, WSL, or Docker). (error log: failed to connect to '192.168.3.15:5555': Connection refused)

But it seems that by default armbian does not support adb as orangpi official roms does. (or I missundestand how to use)

Do you know how to make use of adb shell function (or use any other method) to install the gradio rkllm service on Orangepi 5?

Pelochus commented 4 weeks ago

Sorry, but I haven't tried the Gradio server yet. Perhaps I'll take a look into it by July (currently too busy)