search

v6d-io / v6d

vineyard (v6d): an in-memory immutable data manager. (Project under CNCF, TAG-Storage)
https://v6d.io
Apache License 2.0
818 stars 117 forks source link

Integrate the vineyard llm kv cache with vllm #1839

Open dashanji opened 3 months ago

dashanji commented 3 months ago

Describe your problem

Since we have the python API of llm kv cache, we can integrate it with vllm.

github-actions[bot] commented 2 months ago

/cc @sighingnow, this issus/pr has had no activity for a long time, please help to review the status and assign people to work on it.