search

zilliztech / GPTCache

Semantic cache for LLMs. Fully integrated with LangChain and llama_index.
https://gptcache.readthedocs.io
MIT License
7.25k stars 507 forks source link

[Feature]: Can GPTCache work with FastChat? #631

Open 3togo opened 5 months ago

3togo commented 5 months ago

I am looking for a mean to use GPTCache in FastChat. Any pointer?

SimFG commented 5 months ago

you can try to use the gptcache api, the demo code: https://github.com/zilliztech/GPTCache/blob/main/examples/adapter/api.py