Newest driver from Rockchip implemented support for running LLMs through an API of theirs (or better said, proper support, previous drivers can only run the smallest models). I think it is very interesting to implement this new driver version for accelerating LLMs on Pis.
I'm not very used to building the Linux kernel, but if I can help somehow (testing, for example) let me know.
Newest driver from Rockchip implemented support for running LLMs through an API of theirs (or better said, proper support, previous drivers can only run the smallest models). I think it is very interesting to implement this new driver version for accelerating LLMs on Pis.
I'm not very used to building the Linux kernel, but if I can help somehow (testing, for example) let me know.
Some useful links here: