search

yale-sys / prompt-cache

Modular and structured prompt caching for low-latency LLM inference
MIT License
14 stars 1 forks source link

Initial working version with ms marco #10

Closed shsym closed 8 months ago