Add StreamingLLM to improve streaming performance

lm-sys / FastChat

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

Apache License 2.0

35.65k stars 4.38k forks source link

Open BabyChouSr opened 9 months ago

BabyChouSr commented 9 months ago

Came across this paper on streaming LLMs that may be good for improving the efficiency of streaming.

lujiale621 commented 9 months ago