search

Pelochus / ezrknpu

Easy usage of Rockchip's NPUs found in RK3588 and similar chips
GNU General Public License v3.0
81 stars 4 forks source link

Easy update NPU driver to 0.9.6 #2

Closed Pelochus closed 5 months ago

Pelochus commented 5 months ago

This issue will be used as a thread of me (and anyone that wants to contribute) discovering how to update the driver to 0.9.6 since bigger LLMs (for now 2.7B onwards, but could be 2B onwards) cannot run on driver 0.9.5

For now it is interesting the following:

https://github.com/airockchip/rknn-llm/issues/16

Pelochus commented 5 months ago

First attempt will be integrating this into Armbian and Ubuntu Rockchip. See this:

https://github.com/armbian/linux-rockchip/issues/167

Since Ubuntu Rockchip kernel is forked from Armbian, it should be easy to merge the changes there once it works in Armbian.

Pelochus commented 5 months ago

Partially done!

https://github.com/armbian/linux-rockchip/pull/170

In future Armbian builds (perhaps Ubuntu Rockchip too) will have the updated driver, so at least there are some OSes out there that do not need to update the driver manually.

I'll try to do some images myself for easier testing. Closing this for now