armbian / linux-rockchip

Improved Rockchip Linux
Other
99 stars 142 forks source link

Upgrade NPU driver to 0.9.6 #167

Closed Pelochus closed 2 months ago

Pelochus commented 2 months ago

Newest driver from Rockchip implemented support for running LLMs through an API of theirs (or better said, proper support, previous drivers can only run the smallest models). I think it is very interesting to implement this new driver version for accelerating LLMs on Pis.

I'm not very used to building the Linux kernel, but if I can help somehow (testing, for example) let me know.

Some useful links here:

Pelochus commented 2 months ago

Opened PR, so I'm closing this for now

https://github.com/armbian/linux-rockchip/pull/170