[Feature] 请问turbomind有支持slidding window的计划么？

InternLM / lmdeploy

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

https://lmdeploy.readthedocs.io/en/latest/

Apache License 2.0

4.7k stars 429 forks source link

Open comeby opened 8 months ago

comeby commented 8 months ago

Mistral 里面的 slidding window 对长上下文比较友好，qwen1.5里也有slidding window配置。请问turbomind有支持的计划么？

No response

No response

zhyncs commented 8 months ago

It is likely that support will be added after the LlamaBatch refactoring is completed.

wykvictor commented 4 months ago

Any update? Seems still does not support sliding-window in LMDeploy 0.5.0? Thanks!