whether support sync mongo to kafka cluster?

[sunweiconfidence]

hi @rwynn,

i want to sync mongo replica set change stream to kafka cluster in one server 3 broker or 3 servers 3 broker, whether current version support it? if support, how to configure it? thank you very much.

[rwynn]

hi @sunweiconfidence, each route81 process is able to sync one mongodb cluster to one kafka cluster. You can run as many route81 processes as you like if you need to support multiple clusters of each.

You would configure each process like this...

mongo = "mongodb://[username:password@]host1[:port1][,...hostN[:portN]][/[database][?options]]"
kafka = "hostname:port,hostname2:port2,hostname3:port3"

And run with

route81 -f /path/to/config.toml

[sunweiconfidence]

@rwynn thanks, i need to confirm that you mean that i need to start 3 processes for clusters of 3 servers like below? route81 -f /path/to/config.toml

so i need to write a outer program to start the 3 processes simultaneously? any advise for implement this? thanks.

[rwynn]

@sunweiconfidence, you only need to start separate route81 processes if you have 2 or more mongodb clusters or 2 or more kafka clusters. If you are simply meaning that you have 1 cluster with multiple nodes/servers, then you can specify the nodes as part of a single config as shown above.

Note both connection configs take multiple host/port in the connection string.

[rwynn]

For Kafka, you need to list the comma separated host/port(s) of the brokers

[sunweiconfidence]

@rwynn yes, i only have 1 kafka cluster with multiple nodes/servers for 3 brokers, i understand your meaning now, thanks, i will try it later