zilliztech / GPTCache

Semantic cache for LLMs. Fully integrated with LangChain and llama_index.
https://gptcache.readthedocs.io
MIT License
7.26k stars 507 forks source link

[Bug]: GPTCache initialization fails #635

Closed Chids-gs closed 4 months ago

Chids-gs commented 4 months ago

Current Behavior

We are using GPTCache with Postgres as Cache storage and PGVector as Vector Store. Following is the code that we are using for initialization

`onnx = Onnx()

        init_similar_cache(

                cache_obj=cacheStore.object, post_func=nop,

                data_manager=manager_factory(

                        "postgresql,pgvector",

                vector_params={

                    "dimension": onnx.dimension,

                    "url": <<vector_endpoint>>

                },

                scalar_params={"sql_url": <<sql_endpoint>>}

                ),

                config=Config(similarity_threshold=0.95),

                embedding=onnx,

                evaluation = SearchDistanceEvaluation(),

            )`

gptcache version : 0.1.43

When we restart the application, we are getting the following error `data_manager=manager_factory( ^^^^^^^^^^^^^^^^ File "/home/app/.local/lib/python3.11/site-packages/gptcache/manager/factory.py", line 125, in manager_factory return get_data_manager(s, v, o, e) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/app/.local/lib/python3.11/site-packages/gptcache/manager/factory.py", line 200, in get_data_manager return SSDataManager(cache_base, vector_base, object_base, eviction_base, max_size, clean_size, eviction) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/app/.local/lib/python3.11/site-packages/gptcache/manager/data_manager.py", line 247, in init self.eviction_base.put(self.s.get_ids(deleted=False)) File "/home/app/.local/lib/python3.11/site-packages/gptcache/manager/eviction/memory_cache.py", line 59, in put self._cache[obj] = True


File "/home/app/.local/lib/python3.11/site-packages/cachetools/__init__.py", line 217, in __setitem__
cache_setitem(self, key, value)
File "/home/app/.local/lib/python3.11/site-packages/cachetools/__init__.py", line 79, in __setitem__
self.popitem()
File "/home/app/.local/lib/python3.11/site-packages/gptcache/manager/eviction/memory_cache.py", line 15, in wrapper
wrapper_func(keys)
TypeError: 'NoneType' object is not callable`

After we drop the respective tables and restart the application it is working. Can you let me know how to correct this issue?

### Expected Behavior

Proceed without any issues.

### Steps To Reproduce

_No response_

### Environment

_No response_

### Anything else?

_No response_
SimFG commented 4 months ago

@Chids-gs i try to fix it in my branch, you can try to use it.

Looking forward to your reply!

Chids-gs commented 4 months ago

@SimFG Thanks for the immediate response. I took the version from your branch https://github.com/SimFG/gpt-cache/tree/main and now this issue seems to be fixed.

SimFG commented 4 months ago

@Chids-gs thanks your reply