zilliztech / GPTCache

Semantic cache for LLMs. Fully integrated with LangChain and llama_index.
https://gptcache.readthedocs.io
MIT License
6.89k stars 480 forks source link

[Bug]: default session hit function logic is counter productive #592

Open deven298 opened 6 months ago

deven298 commented 6 months ago

Current Behavior

When the same session is passed in the adapt function. The default session function check_hit_session will return False if the session_id already exists cache_session_ids. Later, the get_scalar_data function will return None if check_hit_session return False for the existing session_id and skip checking the cache data.

Expected Behavior

Ideally, if session_id exists in the cache_session_ids then check_hit_session function should return True so the cached data is further evaluated based on similarity evaluation function and so on.

Steps To Reproduce

No response

Environment

No response

Anything else?

I am building native support using GPTCache package so it is hard to provide reproducing steps. Feel free to comment on the issue and we can connect externally to discuss reproducing steps if the above description is not clear.