search

airockchip / rknn-llm

Other
417 stars 36 forks source link

python deployment code #91

Open openedev opened 2 months ago

openedev commented 2 months ago

I can see only c deployment code. 

https://github.com/airockchip/rknn-llm/tree/main/rkllm-runtime/examples/rkllm_api_demo/src

Do we have any python deployment code? like we have it in toolkit2

https://github.com/airockchip/rknn-toolkit2/tree/master/rknn-toolkit-lite2

Any help?

vincenzodentamaro commented 2 months ago

You have to wrap the c library into a python code as it is done in the examples and call it following the header file interface.

openedev commented 2 months ago

@vincenzodentamaro which example has done the c to python wrap, can you please point?

deployment code mostly has cpp no python.

vincenzodentamaro commented 2 months ago

See https://github.com/airockchip/rknn-llm/blob/main/rkllm-runtime/examples/rkllm_server_demo/rkllm_server/flask_server.py