search

zhr85210078 / node-mongodb-es-connector

nodejs mongodb elasticsearch synchrodata(mongodb和es同步数据)
https://zhr85210078.github.io/node-mongodb-es-connector/#/
MIT License
77 stars 17 forks source link

支持mongos吗 该怎么配置 #30

Open codech opened 4 years ago

codech commented 4 years ago

1、支持mongos吗 该怎么配置 2、几千万的大表同步 是一次性取出所有数据吗

zhr85210078 commented 4 years ago

主要是根据mongo-oplog同步数据 大量数据也是根据配置文件设置的，比如一次取500条，同步到es，循环往复

codech commented 4 years ago

debug: 2020-04-15 17:04:27 | debug | http://10.10.15.94:9200 | responder_post_test | oplog errors: MongoError: can't use 'local' database through mongos Reconnecting... debug: 2020-04-15 17:04:27 | debug | http://10.10.15.94:9200 | responder_post_test | REconnecting 1/times to MongoDB... debug: 2020-04-15 17:04:27 | debug | http://10.10.15.94:9200 | responder_post_test | Start to tail oplog now... debug: 2020-04-15 17:04:27 | debug | http://10.10.15.94:9200 | responder_post_test | oplog errors: MongoError: can't use 'local' database through mongos Reconnecting... debug: 2020-04-15 17:04:27 | debug | http://10.10.15.94:9200 | responder_post_test | REconnecting 2/times to MongoDB... debug: 2020-04-15 17:04:27 | debug | http://10.10.15.94:9200 | responder_post_test | Start to tail oplog now... debug: 2020-04-15 17:04:27 | debug | http://10.10.15.94:9200 | responder_post_test | oplog errors: MongoError: can't use 'local' database through mongos Reconnecting... debug: 2020-04-15 17:04:27 | debug | http://10.10.15.94:9200 | responder_post_test | REconnecting 3/times to MongoDB... debug: 2020-04-15 17:04:27 | debug | http://10.10.15.94:9200 | responder_post_test | Start to tail oplog now... debug: 2020-04-15 17:04:27 | debug | http://10.10.15.94:9200 | responder_post_test | oplog errors: MongoError: can't use 'local' database through mongos Reconnecting... debug: 2020-04-15 17:04:27 | debug | http://10.10.15.94:9200 | responder_post_test | REconnecting 4/times to MongoDB... debug: 2020-04-15 17:04:27 | debug | http://10.10.15.94:9200 | responder_post_test | Start to tail oplog now... debug: 2020-04-15 17:04:27 | debug | http://10.10.15.94:9200 | responder_post_test | oplog errors: MongoError: can't use 'local' database through mongos Reconnecting... debug: 2020-04-15 17:04:27 | debug | http://10.10.15.94:9200 | responder_post_test | REconnecting 5/times to MongoDB... debug: 2020-04-15 17:04:27 | debug | http://10.10.15.94:9200 | responder_post_test | Start to tail oplog now...

mongos 报这个错

zhr85210078 commented 4 years ago

MongoError: can't use 'local' database through mongos Reconnecting... mongodb 必须是replica结构，才能用mongo-oplog

codech commented 4 years ago

建议支持Mongos

mapleFdg commented 4 years ago

建议支持Mongos

有没有支持Mongos的方案？