redis / redis-om-spring

Spring Data Redis extensions for better search, documents models, and more
MIT License
609 stars 94 forks source link

Partial update using Repository.saveAll method #473

Closed shannonantony closed 4 months ago

shannonantony commented 5 months ago

@bsbodden , I am currently loading around 1 million records into Redis using the Repository.saveAll(List) method. I would like to know if there is a way to partially update this object list (only updating 5 out 10 total attributes) in Redis without needing to check for the existence of each object and updating individual attributes.

Your guidance on this matter would be greatly appreciated.

Thank you for your support.

Jay

bsbodden commented 5 months ago

Not that I can think of at the moment. Are your partial updates relying on any logic? Like only update field A if field B has a value C? Or are they just partially populated objects with with and Id set?

We currently have a:

void updateField(T entity, MetamodelField<T, ?> field, Object value);

which allows you update only the given field if you already have an instance of the entity (with an id) and it sends a single JSON.SET with a path (https://redis.io/docs/latest/commands/json.set/) or HSET for the target field.

I could see an extension of this for the QBE (Query by Example) to do something like:

void updateByExample(T entity, Example<S> example)

and we could create one for bulk by example updates as:

void updateAll(Iterable<Example<S>> entities, config);

This will need some flags in config to determine whether to use HSET or HSETNX for hashes and HEXISTS if there is a need to only set it if it already exists. Similarly for JSON.SET. A potential method:

void updateAll(Iterable<S> entities);

could create the examples on behalf of the users and use sensible defaults for the NX/XX flags

shannonantony commented 5 months ago

Thanks @bsbodden for the quick response.

I am trying to update an object (partial) list in batch where I want to update only certain fields while I want to preserve the values for other fields and I want to do in bulk. Looking for some equivalent funtion like JSON.MSET in bulk (pipelined)

thanks

Jay