Closed gyb-uinnova closed 1 year ago
请参考 https://github.com/alibaba/MongoShake/issues/700 提供下相关信息。可能统计有bug
@zhangst 配置文件如下:
conf.version = 10
id = mongoshake
mongoshake
db in the sourcemaster_quorum = false
curl 127.0.0.1:9100
../mongoshake-stat --port=9100
full_sync.http_port = 9101 incr_sync.http_port = 9100
system_profile_port = 9200
log.level = info
log.dir =
log.file = collector.log
log.flush = false
sync_mode = incr
mongo_urls = mongodb://user:password@...1:28017,...2:28017,...3:28017
mongo_cs_url =
mongo_s_url =
mongo_ssl_root_ca_file =
tunnel = direct
mongo_urls
. Iftunnel.address = mongodb://user:password@...1:28017,...2:28017,...3:28017
tunnel.message = raw
tunnel.kafka.partition_number = 1
tunnel.json.format =
tunnel.mongo_ssl_root_ca_file =
mongo_connect_mode = secondaryPreferred
filter.namespace.black = filter.namespace.white =
filter.pass.special.db =
filter.ddl_enable = false
filter.oplog.gids = false
checkpoint.storage.url =
checkpoint.storage.db = mongoshake
checkpoint.storage.collection = ckpt_default
checkpoint.storage.url.mongo_ssl_root_ca_file =
checkpoint.start_position = 1970-01-01T00:00:00Z
transform.namespace =
full_sync.reader.collection_parallel = 6
full_sync.reader.write_document_parallel = 8
full_sync.reader.document_batch_size = 128
full_sync.reader.fetch_batch_size = 8192
full_sync.reader.parallel_thread = 1
full_sync.reader.parallel_index = _id
full_sync.collection_exist_drop = true
full_sync.create_index = none
full_sync.executor.insert_on_dup_update = false
full_sync.executor.filter.orphan_document = false
full_sync.executor.majority_enable = false
incr_sync.mongo_fetch_method = oplog
incr_sync.change_stream.watch_full_document = false
incr_sync.oplog.gids =
collection
if has unique index otherwise use id
.incr_sync.shard_key = collection
incr_sync.shard_by_object_id_whitelist =
incr_sync.worker = 8
incr_sync.tunnel.write_thread = 8
incr_sync.target_delay = 0
incr_sync.worker.batch_queue_size = 64 incr_sync.adaptive.batching_max_size = 1024 incr_sync.fetcher.buffer_capacity = 256 incr_sync.reader.fetch_batch_size = 8192
incr_sync.executor.upsert = false
incr_sync.executor.insert_on_dup_update = false
incr_sync.conflict_write_to = none
incr_sync.executor.majority_enable = false
special.source.db.flag =
遇到这个问题时,源DB的写增量是不是很少? lsn 表示写如目的DB的时间戳的最大值。lsn_ack 表示这个时间戳之前的oplog都已完成写入。这两个都是真正写入目的DB的值。 lsn_ckpt是变量位点的记录,为了增量断点恢复能力,应对长时间没有业务增量的情况,当超过3分钟没有业务增量oplog时,会使用noop等心跳增量来更新lsn_ckpt,用来维持一个可用的位点信息。 所以lsn_ckpt比lsn_ckpt大是正常的。
1.MongoShake的版本:2.8.2 2.源和目的MongoDB的版本:源和目的都是副本集集群,版本4.2.17