redis / redis-vl-python

Redis Vector Library (RedisVL) interfaces with Redis' vector database for realtime semantic search, RAG, and recommendation systems.
https://www.redisvl.com/
MIT License
232 stars 38 forks source link

Support URL based connection to Redis through Sentinel discovery service #213

Open zumo64 opened 2 months ago

zumo64 commented 2 months ago

When using RedisVL from Langchain library the URL based connection is used and doesn't not allow specifying one sentinel nor a list of sentinel agents in the URL.

Getting this error when passing this URL (as documented in the langchain Redis integration documentation)

URL="redis+sentinel://:redis@locahost:8001/mydb/0"
ValueError: Redis URL must specify one of the following schemes (redis://, rediss://, unix://)

We need full support for sentinel discovery service through RedisVL ..

bsbodden commented 2 months ago

@tylerhutcherson this might be a redis-py issue. I check on their repo and nowhere is there a trace of the protocol redis+sentinel substring. But instead there are way to create a programatic Sentinel connection. I will remove the text for the sentinel URL in the integration and add the programatic way, but we should investigate with the redis-py folks about sentinel connection URLs.

FYI, that block for how to connect to Redis was there also in the Community Integration.

tylerhutcherson commented 2 months ago

@bsbodden can you also check langchain-community? I recall someone patched a section for handling redis sentinel, and it was an entire module. If necessary, it's something we could patch in RedisVL. But I want to understand the scope of options to support and impact to the client connection factor in vl first.

As an aside, would it not work to BYO redis client object to LangChain?

bsbodden commented 2 months ago

@bsbodden can you also check langchain-community? I recall someone patched a section for handling redis sentinel, and it was an entire module. If necessary, it's something we could patch in RedisVL. But I want to understand the scope of options to support and impact to the client connection factor in vl first.

As an aside, would it not work to BYO redis client object to LangChain?

That's what @zumo64 is going to try next. I'll check the community integration for that connection URL hack

bsbodden commented 2 months ago

@zumo64 something like:

# establish a Sentinel connection
from redis import Sentinel
sentinel = Sentinel([('localhost', 26379)], socket_timeout=0.1)

index_name = f"test_index_{str(ULID())}"

store = RedisVectorStore.from_texts(
        texts,
        OpenAIEmbeddings(),
        index_name=index_name,
        key_prefix="tst1",
        redis_client=sentinel,
)
# ... work with the store
zumo64 commented 2 months ago

Thanks @bsbodden I have tried but some langchain API calls don't support redis_client as a parameter, example SemanticCache. I will log an issue on that repo as you suggested