The `import/export_kv_cache_buffer` show bad performance

v6d-io / v6d

vineyard (v6d): an in-memory immutable data manager. (Project under CNCF, TAG-Storage)

https://v6d.io

Apache License 2.0

829 stars 122 forks source link

Open dashanji opened 6 months ago

dashanji commented 6 months ago

During the integration with llama.cpp, we find the import_kv_cache_buffer and export_kv_cache_buffer show bad performance as follows.

The Import time includes query vineyard + import_kv_cache_buffer. The Export time includes update vineyard + export_kv_cache_buffer.

github-actions[bot] commented 5 months ago

/cc @sighingnow, this issus/pr has had no activity for a long time, please help to review the status and assign people to work on it.