Finetuning a model in the streaming mode ?

mit-han-lab / streaming-llm

[ICLR 2024] Efficient Streaming Language Models with Attention Sinks

https://arxiv.org/abs/2309.17453

MIT License

6.38k stars 355 forks source link

Finetuning a model in the streaming mode ? #52

Closed MohamedAliRashad closed 9 months ago

MohamedAliRashad commented 9 months ago

I want to ask if i can finetune a normal llm in the streaming mode and if this would be benefiical on results.

Guangxuan-Xiao commented 9 months ago

Hello, StreamingLLM is designed for deploying LLMs with infinite-length inputs without compromising efficiency or performance. It is not intended for training LLMs.