search

v6d-io / v6d

vineyard (v6d): an in-memory immutable data manager. (Project under CNCF, TAG-Storage)
https://v6d.io
Apache License 2.0
829 stars 122 forks source link

llm-cache: cannot match incremental updates #1800

Closed sighingnow closed 7 months ago

sighingnow commented 7 months ago

When using 3 prompts where the first is the prefix of the second one, and the second is further the prefix of the third one:

image

sighingnow commented 7 months ago

Closing as resolved in llama.cpp's integration.