search

airockchip / rknn-llm

Other
258 stars 26 forks source link

在板子上跑server的demo，跑不起来，帮看下什么原因 #53

Open luyu0816 opened 1 month ago

luyu0816 commented 1 month ago

运行日志如下： ~/llm/demo/server_demo$ sudo python flask_server.py --target_platform rk3588 --rkllm_model_path ../../model/minicpm.rkllm =========init....=========== rkllm-runtime version: 1.0.1, rknpu driver version: 0.9.6, platform: RK3588 E RKNN: [14:46:27.960] Cannot load clGetPlatformIDs! Error: could find any OpenCL devices!

chenpingfromgxu commented 1 month ago

我也是同样的问题。不知道如何能处理。

luyu0816 commented 1 month ago

您好！您的邮件我已收到，我会尽快查收，辛苦咯！

chenpingfromgxu commented 1 month ago

../rkllm_server/flask_server.py，第120行应该是False不是True.

.... class RKLLM(object): def init(self, model_path, target_platform): rknnllm_param = RKNNllmParam() rknnllm_param.model_path = bytes(model_path, 'utf-8') if target_platform == "rk3588": rknnllm_param.num_npu_core = 3 elif target_platform == "rk3576": rknnllm_param.num_npu_core = 1 rknnllm_param.max_context_len = 320 rknnllm_param.max_new_tokens = 512 rknnllm_param.top_k = 1 rknnllm_param.top_p = 0.9 rknnllm_param.temperature = 0.8 rknnllm_param.repeat_penalty = 1.1 rknnllm_param.frequency_penalty = 0.0 rknnllm_param.presence_penalty = 0.0 rknnllm_param.mirostat = 0 rknnllm_param.mirostat_tau = 5.0 rknnllm_param.mirostat_eta = 0.1 rknnllm_param.logprobs = False rknnllm_param.top_logprobs = 5 _rknnllm_param.usegpu = False self.handle = RKLLM_Handle_t() .....

fydeos-alex commented 1 month ago

Two reasons caused it:

luyu0816 commented 1 month ago

../rkllm_server/flask_server.py，第120行应该是False不是True.

.... class RKLLM(object): def init(self, model_path, target_platform): rknnllm_param = RKNNllmParam() rknnllm_param.model_path = bytes(model_path, 'utf-8') if target_platform == "rk3588": rknnllm_param.num_npu_core = 3 elif target_platform == "rk3576": rknnllm_param.num_npu_core = 1 rknnllm_param.max_context_len = 320 rknnllm_param.max_new_tokens = 512 rknnllm_param.top_k = 1 rknnllm_param.top_p = 0.9 rknnllm_param.temperature = 0.8 rknnllm_param.repeat_penalty = 1.1 rknnllm_param.frequency_penalty = 0.0 rknnllm_param.presence_penalty = 0.0 rknnllm_param.mirostat = 0 rknnllm_param.mirostat_tau = 5.0 rknnllm_param.mirostat_eta = 0.1 rknnllm_param.logprobs = False rknnllm_param.top_logprobs = 5 _rknnllm_param.usegpu = False self.handle = RKLLM_Handle_t() .....

确实是这个问题，感谢！

aileen5150 commented 2 weeks ago

感谢，确实是120的True导致了此问题