zilliztech / GPTCache

Semantic cache for LLMs. Fully integrated with LangChain and llama_index.
https://gptcache.readthedocs.io
MIT License
7.15k stars 503 forks source link

[DOCS]: Add support for remote databases in GPTCache server config #552

Open echosun1996 opened 11 months ago

echosun1996 commented 11 months ago

Documentation Link

https://github.com/zilliztech/GPTCache/blob/main/examples/README.md#How-to-use-GPTCache-server

Describe the problem

My database service is running in another docker container, I tried to create a separate GPTCache server, but couldn't find the documentation. Here is the code without the GPTCache server:

DSN = f"postgresql+psycopg2://{DB_USER}:{DB_PASSWORD}@{DB_HOST}:{DB_PORT}/{DB_NAME}"
onnx = Onnx()
data_manager = get_data_manager(CacheBase(name='postgresql',sql_url=DSN), VectorBase("pgvector", dimension=onnx.dimension,url=DSN))
cache.init(
    embedding_func=onnx.to_embeddings,
    data_manager=data_manager,
    similarity_evaluation=SearchDistanceEvaluation(),
    )
cache.set_openai_key()

Could you please let me know how to move them in config.yaml to start a GPTCache server?

Anything else?

Here is a sample of config.yaml:

embedding:
    onnx
embedding_config:
    # Set embedding model params here
storage_config:
    data_dir:
        gptcache_data
    manager:
        postgresql,pgvector
    vector_params:
        # Set vector storage related params here
evaluation:
    distance
evaluation_config:
    # Set evaluation metric kws here
pre_function:
    get_prompt
post_function:
    first
config:
    similarity_threshold: 0.8
    # Set other config here
a9raag commented 11 months ago

You can add the scalar parmas under storage config:

embedding:
    onnx
embedding_config:
    # Set embedding model params here
storage_config:
    data_dir:
        gptcache_data
    manager:
        postgresql,pgvector
    vector_params:
        # Set vector storage related params here
    scalar_params:
        sql_url: <URL> 
        table_name: <TABLE_NAME> 

evaluation:
    distance
evaluation_config:
    # Set evaluation metric kws here
pre_function:
    get_prompt
post_function:
    first
config:
    similarity_threshold: 0.8
    # Set other config here