search

zilliztech / GPTCache

Semantic cache for LLMs. Fully integrated with LangChain and llama_index.
https://gptcache.readthedocs.io
MIT License
7.06k stars 500 forks source link

[Feature]: Add support for async connection and operations for PostgreSQL as Cache Storage provider #463

Open andre-meneses-fivestars opened 1 year ago

andre-meneses-fivestars commented 1 year ago

Is your feature request related to a problem? Please describe.

I already use a PostgreSQL database with an async connection in the project where I'm also using GPTCache and would like the connection to that DB for the GPT Cache Storage also to be async.

Describe the solution you'd like.

Cache Base would accept a connection string for PostgreSQL in the format postgresql+asyncpg://{DATABASE_URI} and those connections to the DB would be async.

Describe an alternate solution.

No response

Anything else? (Additional Context)

No response

SimFG commented 1 year ago

Thank you for submitting your proposal. We appreciate your suggestion, which is already part of our plan. However, based on our latest research findings, we have identified that the majority of the time consumption occurs during the model's runtime. Additionally, we must also consider the importance of the storage access time.