search

airockchip / rknn-llm

Other
258 stars 26 forks source link

兼容openai格式的接口 #54

Open smy116 opened 1 month ago

smy116 commented 1 month ago

有办法在本地跑模型后，对外暴露一个兼容openai格式的接口，供ONE-API，沉浸式翻译等应用调用吗

av1d commented 1 month ago

https://github.com/airockchip/rknn-llm/blob/main/rkllm-runtime/examples/rkllm_server_demo/chat_api_flask.py