Bulk Insert Question

[Marty-Jones]

Do you have any suggestions on how I could use a "Batch" to bulk insert items that include tags? I have a case where I need to bulk insert 50-100 items at one time. Each of these items have associated tags with them.

FYI, I will not be running under a Redis cluster for this.

[thepirat000]

There are many ways to do bulk loading, but I think it depends on what are you looking for.

Is this a just once operation, or do you need to implement bulk load as part of your application?

In any case, to be compatible with the default configuration of CachingFramework.Redis, you will need to do two operations per each tagged item:

1. Insert the item on its corresponding key
2. Add the relation tag-key, as a new member into the specialized tag Set key

For example, to insert an object into the key key1 related to the tag tag1

SET "key1" "{\"id\":123,\"name\":\"test\"}"
SADD ":$_tag_$:tag1" "key1"

The default format for the tag keys is by prefixing :$_tag_$: to the tag name. Note each tag is a Redis SET

[Marty-Jones]

Thank you for the information. This was very helpful.

[Marty-Jones]

One more question, in your "SetObjectImpl" method that supports tags, you create a batch and use "SetAddAsync" and "StringSetAsync". These are async operations. I don't see anywhere in the code where you wait for the operations to finish. Is this just something that the batch.Execute() handles for you?

[thepirat000]

I was about to answer that the Execute waits for the operations, but that's not correct.

I'll analyze what's the current behavior for the synchronous SetObject before giving an answer. Looks like the behavior is fire and forget, but I will double check

[thepirat000]

batch.Execute will not wait for the operations to complete, it will just start sending them in bulk. So if you need to explicitly await for the operations, you should use SetObjectAsync.

https://stackoverflow.com/a/61117566/122195

[Marty-Jones]

so does this mean that your current method "SetObject<>" is not a true synchronous call if you call it with tags? It sounds like it will invoke the batch.execute and return so the insert of the cache item entry itself and the the tags will be inserted asyncronously as fire and forget calls (I am assuming fire and forget here).

[thepirat000]

Yes, that's how it's currently implemented