How to convert from Redis to Pinecone?

[fullstackdev610]

I want to replace Redis database to another one, properly Pinecone. In Readme documentation, It mentioned that OpenGPT supports 50+ databases, but no ideas how to switch from Redis to another one. Could you please let me know? Thanks.

[RyanTrojans]

Me too thanks

[CakeCrusher]

@fullstackdev610 you will likely need to rework a lot of the backend

[kevinNejad]

here is a simple solution. keep in mind you still need to add filtering for each agent, depending on how you store vectors in pinecone index.

1. make following changes in backend/packages/gizmo-agent/gizmo_agent/ingest.py

   
   from langchain.vectorstores.pinecone import Pinecone
   import pinecone

pinecone.init(api_key=os.environ["PINECONE_API_KEY"], environment=os.environ["PINECONE_ENV"]) index = pinecone.Index("opengpt")

index_schema = { "tag": [{"name": "namespace"}], }

vstore = Redis(

redis_url=os.environ["REDIS_URL"],

index_name="opengpts",

embedding=OpenAIEmbeddings(),

index_schema=index_schema,

)

vstore = Pinecone(index, OpenAIEmbeddings().embed_query, "random_text")

2. make sure there is no None/NULL in the metadata. you can quickly fix this by making changes in `backend/packages/agent-executor/agent_executor/ingest.py`

def _update_document_metadata(document: Document, namespace: str) -> None: """Mutation in place that adds a namespace to the document metadata.""" document.metadata["namespace"] = namespace

for pincone

document.metadata["source"] = ""  # this is None by default, Pinecone doesn't like None/NULL, so set it to "" or remove it completely, or add the source.

   3. modify create_retriever_tool in `backend/packages/gizmo-agent/gizmo_agent/tools.py`

def get_retrieval_tool(assistant_id: str): return create_retriever_tool( vstore.as_retriever(),

vstore.as_retriever(

    #     search_kwargs={"filter": RedisFilter.tag("namespace") == assistant_id}
    # ),
    "Retriever",
    RETRIEVER_DESCRIPTION,
)
```
here is where you need to change to handle retrieving the docs for specific assistant.