Speed comparasion with exllama and llama.cpp? - Githubissues

mit-han-lab / TinyChatEngine

TinyChatEngine: On-Device LLM Inference Library

https://mit-han-lab.github.io/TinyChatEngine/

MIT License

634 stars 59 forks source link

Speed comparasion with exllama and llama.cpp? #51

Open lucasjinreal opened 10 months ago

lucasjinreal commented 10 months ago

Speed comparasion with exllama and llama.cpp?

RaymondWang0 commented 9 months ago

Hi @lucasjinreal, thank you for your interest in our work. We will compare TinyChatEngine's performance with other frameworks and put the results into our repo asap. Please stay tuned!