Open John-92 opened 2 months ago
[13:27:42 UTC 2024/08/06] [CRIT] (github.com/alibaba/MongoShake/v2/executor.(*Executor).execute:124) Re player-0, executor-0, oplog for namespace[db_tsp_vehicle_data_a] op[u] failed. error type[mongo.CommandError] error[Retryable writes are not supported], logs number[1], firstLog: {" ts":{"T":1722931671,"I":2},"t":17,"h":0,"v":2,"op":"u","ns":"db_tsp_vehicle_data_integrate.otavehicle data","o":[{"Key":"$set","Value":[{"Key":"pushNum","Value":1},{"Key":"pushRet","Value":"DEVICE_CENTE "},{"Key":"responseResult","Value":""},{"Key":"updateBy","Value":null},{"Key":"updateTime", "Value":"2024-08-06T02Z"}]}],"o2":[{"Key":"_id","Value":""}],"lsid":" A","txnNumber":5,"PrevOpTime":"H","ui":{"Subtype":4,"Data":"BbcLqotD"}}
一直以为是Retryable writes are not supported的原因,虽然说mongodb4.0不默认开启Retryable writes,然后在连接上增加retryWrites=true,发现于事无补;然后将目标库的对应的表重命名了,倒是会插入数据,但是checkpoint一直没变,大佬们,求助下
mongoshake 版本2.8.4,配置 conf.version = 10 id = mongoshake master_quorum = false full_sync.http_port = 9101 incr_sync.http_port = 9100 system_profile_port = 9200 log.level = info log.dir = log.file = collector.log log.flush = false
sync_mode = all
mongo_urls = mongodb://xxxx mongo_cs_url = mongo_s_url = mongo_ssl_root_ca_file = tunnel = direct
tunnel.address = mongodb://zxxxxx tunnel.message = raw tunnel.kafka.partition_number = 1 tunnel.json.format = tunnel.mongo_ssl_root_ca_file = mongo_connect_mode = secondaryPreferred
filter.namespace.black =
filter.namespace.white =
filter.pass.special.db =
filter.ddl_enable = true
filter.oplog.gids = false
checkpoint.storage.url =
checkpoint.storage.db = mongoshake
checkpoint.storage.collection = ckpt_default
checkpoint.storage.url.mongo_ssl_root_ca_file =
checkpoint.start_position = 1970-01-01T00:00:00Z
transform.namespace =
full_sync.reader.collection_parallel = 6
full_sync.reader.write_document_parallel = 8
full_sync.reader.document_batch_size = 128
full_sync.reader.fetch_batch_size = 8192
full_sync.reader.parallel_thread = 1
full_sync.reader.parallel_index = _id
full_sync.collection_exist_drop = true
full_sync.create_index = foreground full_sync.executor.insert_on_dup_update = false full_sync.executor.filter.orphan_document = false full_sync.executor.majority_enable = false
incr_sync.mongo_fetch_method = oplog incr_sync.change_stream.watch_full_document = false incr_sync.oplog.gids = incr_sync.shard_key = collection incr_sync.shard_by_object_id_whitelist = incr_sync.worker = 8 incr_sync.tunnel.write_thread = 8 incr_sync.target_delay = 0 incr_sync.worker.batch_queue_size = 64 incr_sync.adaptive.batching_max_size = 1024 incr_sync.fetcher.buffer_capacity = 256 incr_sync.reader.fetch_batch_size = 8192
incr_sync.executor.upsert = false
incr_sync.executor.insert_on_dup_update = false
incr_sync.conflict_write_to = none
incr_sync.executor.majority_enable = false
special.source.db.flag =
一直以为是Retryable writes are not supported的原因,虽然说mongodb4.0不默认开启Retryable writes,然后在连接上增加retryWrites=true,发现于事无补;然后将目标库的对应的表重命名了,倒是会插入数据,但是checkpoint一直没变,大佬们,求助下
对insert操作是可以同步,但是对update的操作,会报错:db.test.findOneAndUpdate( { _id: ObjectId("66b256226945c1c4cc9f603a") }, { $set: { _id: ObjectId("66b256226945c1c4cc9f603a"), vin: "TESTOTA16952187287"}})
对insert操作是可以同步,但是对update的操作,会报错:db.test.findOneAndUpdate( { _id: ObjectId("66b256226945c1c4cc9f603a") }, { $set: vin: "TESTOTA16952187287"}})