search

tulip / oplogtoredis

Publish MongoDB oplog entries to Redis
Apache License 2.0
38 stars 9 forks source link

OplogToRedis: Run writer routines in parallel for each redis client. #75

Closed alex-goodisman closed 4 months ago

alex-goodisman commented 4 months ago

in main, count through each redis client and make a separate chan and coroutine for clientsCount * ordinalCount different entries. make some changes to aggregation in order to accomodate this.

in the tailer, get the [][]chan and for each incoming publication, first index by the shard ordinal (hash of db name), then iterate through the remaining chans (should be one for each redis client) and publish to all of them.

at present, the publisher stream still accepts an array, but now it's only getting one (each).

aranair commented 4 months ago

(small reminder to bump version before merging)