search

mit-han-lab / TinyChatEngine

TinyChatEngine: On-Device LLM Inference Library
https://mit-han-lab.github.io/TinyChatEngine/
MIT License
624 stars 58 forks source link

Support server chat mode #104

Open hyperbolic-c opened 2 months ago

hyperbolic-c commented 2 months ago

Is it possible to add server deploy mode like llama.cpp server? Thanks for your work! !