Add fixed buffer implementation

triton-inference-server / local_cache

Implementation of a local in-memory cache for Triton Inference Server's TRITONCACHE API

BSD 3-Clause "New" or "Revised" License

4 stars 1 forks source link

Add fixed buffer implementation #1

Closed rmccorm4 closed 1 year ago

rmccorm4 commented 1 year ago

TODO

[x] Insert
[x] Lookup
[x] Actual LocalCache implementation with LRU and boost managed buffer
- [x] Intuitive TRITONSERVER_ERROR codes for insert/lookup (already exists, not found, etc.)
[x] Cleanup

rmccorm4 commented 1 year ago

CC @GuanLuo @Tabrizian can't request more than 1 reviewer for private repo, so tagging here. Separate Metrics PR adding back missing cache metrics via Metrics C API to follow.