[Feature]: Is there a Read-Only Mode to GPTCache ? or an alternative solution

[oussamaJmaaa]

Encountered issue :

Currently and to my knowledge , GPTCache automatically inserts new queries into the cache, which may not be desirable in some use cases where only retrieval of cached responses is needed without adding new entries. This makes it harder to control cache growth and manage predictable responses.

Proposed Solution :

Introduce a read-only mode that prevents any new entries from being added to the cache. This could be enabled through a configuration setting, such as cache_read_only=True, which disables cache write operations and allows only retrieval of pre-existing cached responses. If there is a way to achieve this with the current version please let me know .

[SimFG]

yes, the demo code: https://github.com/zilliztech/GPTCache/blob/main/examples/adapter/api.py

[oussamaJmaaa]

Thank you for your quick response but I didn't quite understand where the read-only mode is implemented in the demo you provided. My use case involves using LangChain for a question-answering system, where I want to save a set of FAQs in an SQLite database as the cache. When I run the QA system, it’s not supposed to save new questions or responses to the cache; rather, it should only retrieve from the existing cache if there’s a match. Can you clarify how I can achieve this behavior using GPTCache?

[SimFG]

@oussamaJmaaa I understand what you mean. If you need to implement this, you should implement a read-only cache for langchain based on gptcache. When using gptcache, if you only want to read, you can try using the get method.

from gptcache import cache
from gptcache.adapter.api import put, get
from gptcache.processor.pre import get_prompt

def run_basic():
    cache.init(pre_embedding_func=get_prompt)
    put("hello", "foo")
    print(get("hello"))
    # output: foo

[oussamaJmaaa]

Thanks for your help!