redis / redis-om-spring

Spring Data Redis extensions for better search, documents models, and more
MIT License
596 stars 93 forks source link

Add annotations to support Redis's RediSearch ephemeral indexes #376

Open fcerbell opened 7 months ago

fcerbell commented 7 months ago

Idea :

bsbodden commented 7 months ago

@fcerbell I've been thinking of how to do this in a Spring-way fashion. I'm happy to get on a call to discuss the idea.

bsbodden commented 7 months ago

@fcerbell the documented use case https://redis.com/blog/the-case-for-ephemeral-search/ relies on adding documents manually to the index, which is deprecated I believe. The example relies on data duplication mostly:

At login, the index would be populated with  FT.ADD from the other data source. Nothing special needed here—Search and Query will take care of the document and keys as temporary with no other syntax. Adding the documents will be quick—in the low-single-digit-milliseconds range for most documents. This doesn’t have to be done synchronously, so as a user initially browses the site, the purchase history can be loaded in the background. 

One general note about Search and Query that should be restated, especially in this multi-index context: all document names should be unique across all indexes to prevent key contention at the hash level. Finally, in some circumstances you can save space by using the NOSAVE option on FT.ADD. This will not store the document, but rather just index it, providing you with the document ID only on FT.SEARCH, though this does complicate the result retrieval process.

So big assumption now is that if you have an ephemeral index you also have ephemeral data. What we can do currently is:

bsbodden commented 5 months ago

@fcerbell some issues with Ephemeral indices at the moment...

Posed the question:

Ephemeral Indices:
* I'm implementing a couple of use cases for Ephemeral Indices in Redis OM Spring, first use case is "User already has a large collection of documents/hashes and wants a temporary index narrowed down to a specific user/account"
* Is there a way to prevent the automated dropping of the index from deleting the records associated with the index?

Current answer:

Unfortunately if you using the TTL in this case, it will mimic the FT.DROPINDEX…..DD after x seconds of inactivity. Eventually we should follow the same behaviour here also 

. Adding a ticket for review it. A workaround will be setting this logic in the application using FT.CREATE + FT.DROPINDEX (without DD)