tulip / oplogtoredis

Publish MongoDB oplog entries to Redis
Apache License 2.0
38 stars 9 forks source link

Open up for sending the message to more than two channels #32

Closed SimonSimCity closed 2 years ago

SimonSimCity commented 4 years ago

The design as currently is only allows sending a message into two channels:

This PR is a first improvement of this design towards supporting custom channels, as used when fine-tuning the redis-oplog package: https://github.com/cult-of-coders/redis-oplog/blob/master/docs/finetuning.md

It doesn't change anything on the actual result, but allows the processor to define a list of channels instead of expecting two channels.

The publisher receives the Publication documents and concatenates the channels, which Redis (using a LUA script) then splits again to get the list of channels in which it can send the message to.

Decision taken: Since the arguments, passed to LUA only accepts strings, we have to concatenate the list of channels to a string - means I had to choose a separation character. Since MongoDB does neither allow the $ sign in database nor collection names, I chose it (https://docs.mongodb.com/manual/reference/limits/#Restriction-on-Collection-Names).