Pelochus
commented
2 months ago Not only that but Llama 2 is no longer supported. Sigh, for now I'll not reconvert broken models until rkllm settles a bit more.
Your other error is my fault for not properly rebasing with the new version, I already solved it, do a git pull and that should do the trick.
Well, Rockchip changed enough code to break previously converted models. I tried running your llama 7b and 13b and both returned an error.
rkllm llama2-chat-7b-hf.rkllm RKLLM starting, please wait... rkllm-runtime version: 1.0.1 Warning: The model version is too old, please use the latest toolkit to reconvert the model! The model target_platform does not match! RKLLM: failed to load model 'llama2-chat-7b-hf.rkllm'!!! RKLLM init success!I also received this error when I first attempted to install the updated version:
/home/kh/Documents/ezrknn-llm/rkllm-runtime/examples/rkllm_api_demo/src/main.cpp: In function ‘void callback(RKLLMResult*, void*, LLMCallState)’: /home/kh/Documents/ezrknn-llm/rkllm-runtime/examples/rkllm_api_demo/src/main.cpp:57:1: error: version control conflict marker in file 57 | <<<<<<< HEAD:rkllm-runtime/example/src/main.cpp | ^~~~~~~ /home/kh/Documents/ezrknn-llm/rkllm-runtime/examples/rkllm_api_demo/src/main.cpp:59:1: error: version control conflict marker in file 59 | ======= | ^~~~~~~ /home/kh/Documents/ezrknn-llm/rkllm-runtime/examples/rkllm_api_demo/src/main.cpp:61:1: error: version control conflict marker in file 61 | >>>>>>> airockchip-main:rkllm-runtime/examples/rkllm_api_demo/src/main.cpp | ^~~~~~~ /home/kh/Documents/ezrknn-llm/rkllm-runtime/examples/rkllm_api_demo/src/main.cpp: In function ‘int main(int, char**)’: /home/kh/Documents/ezrknn-llm/rkllm-runtime/examples/rkllm_api_demo/src/main.cpp:83:1: error: version control conflict marker in file 83 | <<<<<<< HEAD:rkllm-runtime/example/src/main.cpp | ^~~~~~~ /home/kh/Documents/ezrknn-llm/rkllm-runtime/examples/rkllm_api_demo/src/main.cpp:85:3: error: expected primary-expression before ‘==’ token 85 | ======= | ^~ /home/kh/Documents/ezrknn-llm/rkllm-runtime/examples/rkllm_api_demo/src/main.cpp:85:5: error: expected primary-expression before ‘==’ token 85 | ======= | ^~ /home/kh/Documents/ezrknn-llm/rkllm-runtime/examples/rkllm_api_demo/src/main.cpp:85:7: error: expected primary-expression before ‘=’ token 85 | ======= | ^ /home/kh/Documents/ezrknn-llm/rkllm-runtime/examples/rkllm_api_demo/src/main.cpp:90:1: error: version control conflict marker in file 90 | >>>>>>> airockchip-main:rkllm-runtime/examples/rkllm_api_demo/src/main.cpp | ^~~~~~~ /home/kh/Documents/ezrknn-llm/rkllm-runtime/examples/rkllm_api_demo/src/main.cpp:97:31: error: ‘struct RKLLMParam’ has no member named ‘target_platform’ 97 | pre_input.push_back(param.target_platform); | ^~~~~~~~~~~~~~~ make[2]: *** [CMakeFiles/llm_demo.dir/build.make:76: CMakeFiles/llm_demo.dir/src/main.cpp.o] Error 1 make[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/llm_demo.dir/all] Error 2 make: *** [Makefile:91: all] Error 2