search

alibaba / MongoShake

MongoShake is a universal data replication platform based on MongoDB's oplog. Redundant replication and active-active replication are two most important functions. 基于mongodb oplog的集群复制工具，可以满足迁移和同步的需求，进一步实现灾备和多活功能。
GNU General Public License v3.0
1.72k stars 441 forks source link

mongodb2.4.9一台副本服务器上，同步到monogdb4.2 出错 #492

Closed 49566258 closed 3 years ago

49566258 commented 3 years ago

RS5:SECONDARY> rs.config() { "_id" : "RS5", "version" : 6, "members" : [ { "_id" : 0, "host" : "10.168.168.18:27002" }, { "_id" : 3, "host" : "10.168.170.179 :37004", "arbiterOnly" : true }, { "_id" : 4, "host" : "10.168.168.17:27002" } ] } [root@ceph-170-186 mongo-shake-v2.4.16]# cat collector.conf conf.version = 6 id = mongoshake master_quorum = false full_sync.http_port = 9101 incr_sync.http_port = 9100 system_profile_port = 9200 log.level = info log.dir =/tmp/ log.file = collector.log log.flush = true

log.flush = false

sync_mode = full mongo_urls = mongodb://10.168.168.17:27002 tunnel = direct tunnel.address = mongodb://10.168.170.187:27017 tunnel.message = raw mongo_connect_mode = primary filter.namespace.white = db_news_rank filter.ddl_enable = true checkpoint.storage.url = mongodb://10.168.170.187:27017 checkpoint.storage.db = mongoshake checkpoint.storage.collection = ckpt_default checkpoint.start_position = 1970-01-01T00:00:00Z full_sync.reader.collection_parallel = 6 full_sync.reader.write_document_parallel = 8 full_sync.reader.document_batch_size = 128 full_sync.reader.read_document_count = 0 full_sync.collection_exist_drop = false full_sync.create_index = none full_sync.executor.insert_on_dup_update = false full_sync.executor.filter.orphan_document = false full_sync.executor.majority_enable = false incr_sync.mongo_fetch_method = oplog incr_sync.change_stream.watch_full_document = false incr_sync.shard_key = collection incr_sync.worker = 8 incr_sync.worker.oplog_compressor = none incr_sync.target_delay = 0 incr_sync.worker.batch_queue_size = 64 incr_sync.adaptive.batching_max_size = 1024 incr_sync.fetcher.buffer_capacity = 256 incr_sync.executor.upsert = false incr_sync.executor.insert_on_dup_update = false incr_sync.conflict_write_to = none incr_sync.executor.majority_enable = false [root@ceph-170-186 mongo-shake-v2.4.16]# ./collector.linux -conf=collector.conf -verbose [2020/11/27 13:55:36 CST] [WARN]

\ \ __ | \ \ / __-=O'/|O'/| \ MongoShake, Here we go !! _____\ / | / ) / / '/-==_ /|/=-| -GM / Alibaba Cloud / * \ | | / / (o)

if you have any problem, please visit https://github.com/alibaba/MongoShake/wiki/FAQ

[2020/11/27 13:55:36 CST] [INFO] New session to mongodb://10.168.170.187:27017 successfully [2020/11/27 13:55:36 CST] [INFO] Close session with mongodb://10.168.170.187:27017 [2020/11/27 13:55:36 CST] [INFO] New session to mongodb://10.168.168.17:27002 successfully [2020/11/27 13:55:36 CST] [INFO] Close session with mongodb://10.168.168.17:27002 [2020/11/27 13:55:36 CST] [INFO] Collector startup. shard_by[collection] gids[[]] [2020/11/27 13:55:36 CST] [INFO] Collector configuration {"ConfVersion":6,"Id":"mongoshake","MasterQuorum":false,"FullSyncHTTPListenPort":9101,"IncrSyncHTTPListenPort":9100,"SystemProfilePort":9200,"LogLevel":"info","LogDirectory":"/tmp/","LogFileName":"collector.log","LogFlush":true,"SyncMode":"full","MongoUrls":["mongodb://10.168.168.17:27002"],"MongoCsUrl":"","MongoSUrl":"","MongoConnectMode":"primary","Tunnel":"direct","TunnelAddress":["mongodb://10.168.170.187:27017"],"TunnelMessage":"raw","FilterNamespaceBlack":null,"FilterNamespaceWhite":["db_news_rank"],"FilterPassSpecialDb":null,"FilterDDLEnable":true,"CheckpointStorageUrl":"mongodb://10.168.170.187:27017","CheckpointStorageDb":"mongoshake","CheckpointStorageCollection":"ckpt_default","CheckpointStartPosition":1,"TransformNamespace":null,"FullSyncReaderCollectionParallel":6,"FullSyncReaderWriteDocumentParallel":8,"FullSyncReaderReadDocumentCount":0,"FullSyncReaderDocumentBatchSize":128,"FullSyncCollectionDrop":false,"FullSyncCreateIndex":"none","FullSyncReaderOplogStoreDisk":false,"FullSyncReaderOplogStoreDiskMaxSize":256000,"FullSyncExecutorInsertOnDupUpdate":false,"FullSyncExecutorFilterOrphanDocument":false,"FullSyncExecutorMajorityEnable":false,"IncrSyncMongoFetchMethod":"oplog","IncrSyncChangeStreamWatchFullDocument":false,"IncrSyncOplogGIDS":null,"IncrSyncShardKey":"collection","IncrSyncShardByObjectIdWhiteList":null,"IncrSyncWorker":8,"IncrSyncTargetDelay":0,"IncrSyncWorkerOplogCompressor":"none","IncrSyncWorkerBatchQueueSize":64,"IncrSyncAdaptiveBatchingMaxSize":1024,"IncrSyncFetcherBufferCapacity":256,"IncrSyncExecutorUpsert":false,"IncrSyncExecutorInsertOnDupUpdate":false,"IncrSyncConflictWriteTo":"none","IncrSyncExecutorMajorityEnable":false,"CheckpointStorage":"database","CheckpointInterval":5000,"FullSyncExecutorDebug":false,"IncrSyncDBRef":false,"IncrSyncExecutor":1,"IncrSyncExecutorDebug":false,"IncrSyncReaderDebug":"","IncrSyncCollisionEnable":false,"IncrSyncReaderBufferTime":1,"Version":"improve-2.4.16,7bd0a9be075037779760fe8704ea2e95d25d908b,release,go1.10.3,2020-11-04_14:33:56","IncrSyncTunnel":"","IncrSyncTunnelAddress":null,"IncrSyncTunnelMessage":"","HTTPListenPort":0,"SystemProfile":0} [2020/11/27 13:55:36 CST] [INFO] start running with mode[full], fullBeginTs[0[0, 0]] [2020/11/27 13:55:36 CST] [INFO] source is replica or mongos, no need to fetching chunk map [2020/11/27 13:55:36 CST] [INFO] New session to mongodb://10.168.168.17:27002 successfully [2020/11/27 13:55:36 CST] [INFO] Close session with mongodb://10.168.168.17:27002 [2020/11/27 13:55:36 CST] [INFO] all namespace: map[{db_news_rank t_news_ban}:{} {db_news_rank t_rank_set}:{} {db_news_rank t_time_hit}:{} {db_news_rank t_time_hit_min}:{} {db_news_rank zt_news_hit}:{} {db_news_rank zt_rank_hit}:{}] [2020/11/27 13:55:36 CST] [INFO] New session to mongodb://10.168.170.187:27017 successfully [2020/11/27 13:55:36 CST] [INFO] replication from [replica] to [replica] [2020/11/27 13:55:36 CST] [INFO] document syncer-0 do replication for url=mongodb://10.168.168.17:27002 [2020/11/27 13:55:36 CST] [INFO] New session to mongodb://10.168.168.17:27002 successfully [2020/11/27 13:55:36 CST] [INFO] Close session with mongodb://10.168.168.17:27002 [2020/11/27 13:55:36 CST] [INFO] DBSyncer id[0] source[mongodb://10.168.168.17:27002] target[mongodb://10.168.170.187:27017] startTime[2020-11-27 13:55:36.848518587 +0800 CST m=+0.027989805] collExecutor-5 sync ns {db_news_rank t_news_ban} to {db_news_rank t_news_ban} begin [2020/11/27 13:55:36 CST] [INFO] DBSyncer id[0] source[mongodb://10.168.168.17:27002] target[mongodb://10.168.170.187:27017] startTime[2020-11-27 13:55:36.848518587 +0800 CST m=+0.027989805] collExecutor-3 sync ns {db_news_rank t_time_hit} to {db_news_rank t_time_hit} begin [2020/11/27 13:55:36 CST] [INFO] DBSyncer id[0] source[mongodb://10.168.168.17:27002] target[mongodb://10.168.170.187:27017] startTime[2020-11-27 13:55:36.848518587 +0800 CST m=+0.027989805] collExecutor-2 sync ns {db_news_rank t_rank_set} to {db_news_rank t_rank_set} begin [2020/11/27 13:55:36 CST] [INFO] DBSyncer id[0] source[mongodb://10.168.168.17:27002] target[mongodb://10.168.170.187:27017] startTime[2020-11-27 13:55:36.848518587 +0800 CST m=+0.027989805] collExecutor-4 sync ns {db_news_rank t_time_hit_min} to {db_news_rank t_time_hit_min} begin [2020/11/27 13:55:36 CST] [INFO] DBSyncer id[0] source[mongodb://10.168.168.17:27002] target[mongodb://10.168.170.187:27017] startTime[2020-11-27 13:55:36.848518587 +0800 CST m=+0.027989805] collExecutor-1 sync ns {db_news_rank zt_rank_hit} to {db_news_rank zt_rank_hit} begin [2020/11/27 13:55:36 CST] [INFO] DBSyncer id[0] source[mongodb://10.168.168.17:27002] target[mongodb://10.168.170.187:27017] startTime[2020-11-27 13:55:36.848518587 +0800 CST m=+0.027989805] collExecutor-0 sync ns {db_news_rank zt_news_hit} to {db_news_rank zt_news_hit} begin [2020/11/27 13:55:36 CST] [INFO] New session to mongodb://10.168.170.187:27017 successfully [2020/11/27 13:55:36 CST] [INFO] New session to mongodb://10.168.170.187:27017 successfully [2020/11/27 13:55:36 CST] [INFO] New session to mongodb://10.168.170.187:27017 successfully [2020/11/27 13:55:36 CST] [INFO] New session to mongodb://10.168.170.187:27017 successfully [2020/11/27 13:55:36 CST] [INFO] New session to mongodb://10.168.170.187:27017 successfully [2020/11/27 13:55:36 CST] [INFO] New session to mongodb://10.168.170.187:27017 successfully [2020/11/27 13:55:36 CST] [INFO] New session to mongodb://10.168.168.17:27002 successfully [2020/11/27 13:55:36 CST] [INFO] New session to mongodb://10.168.168.17:27002 successfully [2020/11/27 13:55:36 CST] [INFO] New session to mongodb://10.168.168.17:27002 successfully [2020/11/27 13:55:36 CST] [INFO] New session to mongodb://10.168.168.17:27002 successfully [2020/11/27 13:55:36 CST] [INFO] New session to mongodb://10.168.168.17:27002 successfully [2020/11/27 13:55:36 CST] [INFO] New session to mongodb://10.168.168.17:27002 successfully [2020/11/27 13:55:36 CST] [INFO] splitter[DocumentSplitter src[mongodb://10.168.168.17:27002] ns[{db_news_rank t_time_hit}] count[1754965] pieceSize[0] pieceNumber[1]] disable split or no need [2020/11/27 13:55:36 CST] [INFO] splitter[DocumentSplitter src[mongodb://10.168.168.17:27002] ns[{db_news_rank t_news_ban}] count[746] pieceSize[0] pieceNumber[1]] disable split or no need [2020/11/27 13:55:36 CST] [INFO] splitter[DocumentSplitter src[mongodb://10.168.168.17:27002] ns[{db_news_rank t_rank_set}] count[4] pieceSize[0] pieceNumber[1]] disable split or no need [2020/11/27 13:55:36 CST] [INFO] splitter[DocumentSplitter src[mongodb://10.168.168.17:27002] ns[{db_news_rank t_rank_set}] count[4] pieceSize[0] pieceNumber[1]] exits [2020/11/27 13:55:36 CST] [INFO] splitter[DocumentSplitter src[mongodb://10.168.168.17:27002] ns[{db_news_rank t_time_hit}] count[1754965] pieceSize[0] pieceNumber[1]] exits [2020/11/27 13:55:36 CST] [INFO] splitter[DocumentSplitter src[mongodb://10.168.168.17:27002] ns[{db_news_rank t_news_ban}] count[746] pieceSize[0] pieceNumber[1]] exits [2020/11/27 13:55:36 CST] [INFO] splitter[DocumentSplitter src[mongodb://10.168.168.17:27002] ns[{db_news_rank zt_rank_hit}] count[83601] pieceSize[0] pieceNumber[1]] disable split or no need [2020/11/27 13:55:36 CST] [INFO] splitter[DocumentSplitter src[mongodb://10.168.168.17:27002] ns[{db_news_rank zt_rank_hit}] count[83601] pieceSize[0] pieceNumber[1]] exits [2020/11/27 13:55:36 CST] [INFO] splitter[DocumentSplitter src[mongodb://10.168.168.17:27002] ns[{db_news_rank t_time_hit_min}] count[104434] pieceSize[0] pieceNumber[1]] disable split or no need [2020/11/27 13:55:36 CST] [INFO] splitter[DocumentSplitter src[mongodb://10.168.168.17:27002] ns[{db_news_rank zt_news_hit}] count[382] pieceSize[0] pieceNumber[1]] disable split or no need [2020/11/27 13:55:36 CST] [INFO] splitter[DocumentSplitter src[mongodb://10.168.168.17:27002] ns[{db_news_rank t_time_hit_min}] count[104434] pieceSize[0] pieceNumber[1]] exits [2020/11/27 13:55:36 CST] [INFO] splitter[DocumentSplitter src[mongodb://10.168.168.17:27002] ns[{db_news_rank zt_news_hit}] count[382] pieceSize[0] pieceNumber[1]] exits [2020/11/27 13:55:41 CST] [INFO] [name=RS5, stage=full, get=0, tps=0] [2020/11/27 13:55:46 CST] [INFO] [name=RS5, stage=full, get=0, tps=0] [2020/11/27 13:55:51 CST] [INFO] [name=RS5, stage=full, get=0, tps=0] [2020/11/27 13:55:56 CST] [INFO] [name=RS5, stage=full, get=0, tps=0] [2020/11/27 13:56:01 CST] [INFO] [name=RS5, stage=full, get=0, tps=0] [2020/11/27 13:56:06 CST] [INFO] [name=RS5, stage=full, get=0, tps=0] panic: send on closed channel

goroutine 257 [running]: vendor/github.com/vinllen/log4go.(ConsoleLogWriter).LogWrite(0xc42000c780, 0xc42036e080) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/log4go/termlog.go:41 +0x43 vendor/github.com/vinllen/log4go.Logger.intLogf(0xc42001f860, 0x7, 0xcd616c, 0x2, 0xc42041df98, 0x1, 0x1) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/log4go/log4go.go:223 +0x289 vendor/github.com/vinllen/log4go.Crashf(0xcd616c, 0x2, 0xc42041df98, 0x1, 0x1) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/log4go/wrapper.go:44 +0x6c mongoshake/collector/docsyncer.(DBSyncer).collectionSync.func1(0xc42040a1e0, 0xc420267360, 0xc42023e280, 0xc420558c20, 0xc4204a68a0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_syncer.go:422 +0xeb created by mongoshake/collector/docsyncer.(*DBSyncer).collectionSync /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_syncer.go:414 +0x2cc [root@ceph-170-186 mongo-shake-v2.4.16]#

49566258 commented 3 years ago

再次运行会报ping mongodb timeout，如下 [2020/11/27 14:11:16 CST] [INFO] New session to mongodb://10.168.168.17:27002 successfully [2020/11/27 14:11:16 CST] [INFO] New session to mongodb://10.168.168.17:27002 successfully [2020/11/27 14:11:16 CST] [INFO] New session to mongodb://10.168.168.17:27002 successfully [2020/11/27 14:11:16 CST] [INFO] New session to mongodb://10.168.168.17:27002 successfully [2020/11/27 14:11:16 CST] [INFO] splitter[DocumentSplitter src[mongodb://10.168.168.17:27002] ns[{db_news_rank t_news_ban}] count[746] pieceSize[0] pieceNumber[1]] disable split or no need [2020/11/27 14:11:16 CST] [INFO] splitter[DocumentSplitter src[mongodb://10.168.168.17:27002] ns[{db_news_rank t_news_ban}] count[746] pieceSize[0] pieceNumber[1]] exits [2020/11/27 14:11:16 CST] [INFO] New session to mongodb://10.168.168.17:27002 successfully [2020/11/27 14:11:16 CST] [INFO] splitter[DocumentSplitter src[mongodb://10.168.168.17:27002] ns[{db_news_rank zt_rank_hit}] count[83601] pieceSize[0] pieceNumber[1]] disable split or no need [2020/11/27 14:11:16 CST] [INFO] splitter[DocumentSplitter src[mongodb://10.168.168.17:27002] ns[{db_news_rank zt_rank_hit}] count[83601] pieceSize[0] pieceNumber[1]] exits [2020/11/27 14:11:16 CST] [INFO] New session to mongodb://10.168.168.17:27002 successfully [2020/11/27 14:11:16 CST] [INFO] splitter[DocumentSplitter src[mongodb://10.168.168.17:27002] ns[{db_news_rank t_time_hit}] count[1756562] pieceSize[0] pieceNumber[1]] disable split or no need [2020/11/27 14:11:16 CST] [INFO] splitter[DocumentSplitter src[mongodb://10.168.168.17:27002] ns[{db_news_rank t_time_hit}] count[1756562] pieceSize[0] pieceNumber[1]] exits [2020/11/27 14:11:16 CST] [INFO] splitter[DocumentSplitter src[mongodb://10.168.168.17:27002] ns[{db_news_rank t_time_hit_min}] count[107367] pieceSize[0] pieceNumber[1]] disable split or no need [2020/11/27 14:11:16 CST] [INFO] splitter[DocumentSplitter src[mongodb://10.168.168.17:27002] ns[{db_news_rank t_time_hit_min}] count[107367] pieceSize[0] pieceNumber[1]] exits [2020/11/27 14:11:16 CST] [INFO] splitter[DocumentSplitter src[mongodb://10.168.168.17:27002] ns[{db_news_rank t_rank_set}] count[4] pieceSize[0] pieceNumber[1]] disable split or no need [2020/11/27 14:11:16 CST] [INFO] splitter[DocumentSplitter src[mongodb://10.168.168.17:27002] ns[{db_news_rank t_rank_set}] count[4] pieceSize[0] pieceNumber[1]] exits [2020/11/27 14:11:16 CST] [INFO] splitter[DocumentSplitter src[mongodb://10.168.168.17:27002] ns[{db_news_rank zt_news_hit}] count[382] pieceSize[0] pieceNumber[1]] disable split or no need [2020/11/27 14:11:16 CST] [INFO] splitter[DocumentSplitter src[mongodb://10.168.168.17:27002] ns[{db_news_rank zt_news_hit}] count[382] pieceSize[0] pieceNumber[1]] exits [2020/11/27 14:11:21 CST] [INFO] [name=RS5, stage=full, get=0, tps=0] [2020/11/27 14:11:26 CST] [INFO] [name=RS5, stage=full, get=0, tps=0] [2020/11/27 14:11:31 CST] [INFO] [name=RS5, stage=full, get=0, tps=0] [2020/11/27 14:11:36 CST] [INFO] [name=RS5, stage=full, get=0, tps=0] [2020/11/27 14:11:41 CST] [INFO] [name=RS5, stage=full, get=0, tps=0] [2020/11/27 14:11:46 CST] [INFO] [name=RS5, stage=full, get=0, tps=0] [2020/11/27 14:11:46 CST] [CRIT] splitter reader[DocumentReader src[mongodb://10.168.168.17:27002] ns[{db_news_rank t_news_ban}] query[map[]]] get next document failed: ping to mongodb://10.168.168.17:27002 failed: server selection error: server selection timeout, current topology: { Type: Unknown, Servers: [{ Addr: 10.168.168.17:27002, Type: RSSecondary, State: Connected, Average RTT: 535234 }, ] } panic: send on closed channel

goroutine 374 [running]: vendor/github.com/vinllen/log4go.(ConsoleLogWriter).LogWrite(0xc42000c780, 0xc42031a200) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/log4go/termlog.go:41 +0x43 vendor/github.com/vinllen/log4go.Logger.intLogf(0xc42001f860, 0x7, 0xcd616c, 0x2, 0xc42061df98, 0x1, 0x1) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/log4go/log4go.go:223 +0x289 vendor/github.com/vinllen/log4go.Crashf(0xcd616c, 0x2, 0xc42061df98, 0x1, 0x1) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/log4go/wrapper.go:44 +0x6c mongoshake/collector/docsyncer.(DBSyncer).collectionSync.func1(0xc42049e060, 0xc4202d88c0, 0xc4200fa300, 0xc42020cf80, 0xc420214f10) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_syncer.go:422 +0xeb created by mongoshake/collector/docsyncer.(*DBSyncer).collectionSync /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_syncer.go:414 +0x2cc

goroutine 245 [chan receive]: mongoshake/collector/docsyncer.(DocExecutor).start(0xc4202aee10) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:148 +0x95 created by mongoshake/collector/docsyncer.(CollectionExecutor).Start /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:75 +0x161

goroutine 246 [chan receive]: mongoshake/collector/docsyncer.(DocExecutor).start(0xc4202aee40) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:148 +0x95 created by mongoshake/collector/docsyncer.(CollectionExecutor).Start /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:75 +0x161

goroutine 247 [chan receive]: mongoshake/collector/docsyncer.(DocExecutor).start(0xc4202aee70) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:148 +0x95 created by mongoshake/collector/docsyncer.(CollectionExecutor).Start /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:75 +0x161

goroutine 212 [IO wait]: internal/poll.runtime_pollWait(0x7f334a2bf470, 0x72, 0xc420300c88) /home/zhuzhao.cx/go/src/runtime/netpoll.go:173 +0x57 internal/poll.(pollDesc).wait(0xc420550098, 0x72, 0xffffffffffffff00, 0xe21b60, 0x139ddd0) /home/zhuzhao.cx/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b internal/poll.(pollDesc).waitRead(0xc420550098, 0xc420540200, 0x10, 0x10) /home/zhuzhao.cx/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d internal/poll.(FD).Read(0xc420550080, 0xc4205402c0, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/internal/poll/fd_unix.go:157 +0x17d net.(netFD).Read(0xc420550080, 0xc4205402c0, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/net/fd_unix.go:202 +0x4f net.(conn).Read(0xc420210088, 0xc4205402c0, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/net/net.go:176 +0x6a vendor/github.com/vinllen/mgo.fill(0xe2a140, 0xc420210088, 0xc4205402c0, 0x10, 0x10, 0x0, 0x11) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:601 +0x53 vendor/github.com/vinllen/mgo.(mongoSocket).readLoop(0xc4202ca7e0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:619 +0x122 created by vendor/github.com/vinllen/mgo.newSocket /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:211 +0x1f5

goroutine 248 [chan receive]: mongoshake/collector/docsyncer.(DocExecutor).start(0xc4202aeea0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:148 +0x95 created by mongoshake/collector/docsyncer.(CollectionExecutor).Start /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:75 +0x161

goroutine 249 [chan receive]: mongoshake/collector/docsyncer.(DocExecutor).start(0xc4202aeed0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:148 +0x95 created by mongoshake/collector/docsyncer.(CollectionExecutor).Start /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:75 +0x161

goroutine 250 [select]: vendor/github.com/vinllen/mgo.(*mongoCluster).syncServersLoop(0xc4200e3d40) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/cluster.go:431 +0x31a created by vendor/github.com/vinllen/mgo.newCluster /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/cluster.go:83 +0x1ae

goroutine 191 [IO wait]: internal/poll.runtime_pollWait(0x7f334a2bf3a0, 0x72, 0xc420301c88) /home/zhuzhao.cx/go/src/runtime/netpoll.go:173 +0x57 internal/poll.(pollDesc).wait(0xc4202e4798, 0x72, 0xffffffffffffff00, 0xe21b60, 0x139ddd0) /home/zhuzhao.cx/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b internal/poll.(pollDesc).waitRead(0xc4202e4798, 0xc42034e300, 0x10, 0x10) /home/zhuzhao.cx/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d internal/poll.(FD).Read(0xc4202e4780, 0xc42034e350, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/internal/poll/fd_unix.go:157 +0x17d net.(netFD).Read(0xc4202e4780, 0xc42034e350, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/net/fd_unix.go:202 +0x4f net.(conn).Read(0xc42017a088, 0xc42034e350, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/net/net.go:176 +0x6a vendor/github.com/vinllen/mgo.fill(0xe2a140, 0xc42017a088, 0xc42034e350, 0x10, 0x10, 0x0, 0x11) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:601 +0x53 vendor/github.com/vinllen/mgo.(mongoSocket).readLoop(0xc4204b6120) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:619 +0x122 created by vendor/github.com/vinllen/mgo.newSocket /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:211 +0x1f5

goroutine 155 [chan receive]: mongoshake/collector/docsyncer.(DocExecutor).start(0xc420200930) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:148 +0x95 created by mongoshake/collector/docsyncer.(CollectionExecutor).Start /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:75 +0x161

goroutine 300 [sleep]: time.Sleep(0x37e11d600) /home/zhuzhao.cx/go/src/runtime/time.go:102 +0x166 vendor/github.com/vinllen/mgo.(*mongoServer).pinger(0xc4202f4a50, 0xc420212801) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/server.go:426 +0x4b3 created by vendor/github.com/vinllen/mgo.newServer /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/server.go:107 +0x186

goroutine 156 [chan receive]: mongoshake/collector/docsyncer.(DocExecutor).start(0xc420200960) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:148 +0x95 created by mongoshake/collector/docsyncer.(CollectionExecutor).Start /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:75 +0x161

goroutine 157 [chan receive]: mongoshake/collector/docsyncer.(DocExecutor).start(0xc420200990) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:148 +0x95 created by mongoshake/collector/docsyncer.(CollectionExecutor).Start /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:75 +0x161

goroutine 252 [sleep]: time.Sleep(0x37e11d600) /home/zhuzhao.cx/go/src/runtime/time.go:102 +0x166 vendor/github.com/vinllen/mgo.(*mongoServer).pinger(0xc4202c5b30, 0xc4203b7101) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/server.go:426 +0x4b3 created by vendor/github.com/vinllen/mgo.newServer /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/server.go:107 +0x186

goroutine 158 [chan receive]: mongoshake/collector/docsyncer.(DocExecutor).start(0xc4202009c0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:148 +0x95 created by mongoshake/collector/docsyncer.(CollectionExecutor).Start /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:75 +0x161

goroutine 159 [chan receive]: mongoshake/collector/docsyncer.(DocExecutor).start(0xc4202009f0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:148 +0x95 created by mongoshake/collector/docsyncer.(CollectionExecutor).Start /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:75 +0x161

goroutine 160 [chan receive]: mongoshake/collector/docsyncer.(DocExecutor).start(0xc420200a20) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:148 +0x95 created by mongoshake/collector/docsyncer.(CollectionExecutor).Start /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:75 +0x161

goroutine 161 [select]: vendor/github.com/vinllen/mgo.(*mongoCluster).syncServersLoop(0xc42024e360) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/cluster.go:431 +0x31a created by vendor/github.com/vinllen/mgo.newCluster /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/cluster.go:83 +0x1ae

goroutine 275 [sleep]: time.Sleep(0x37e11d600) /home/zhuzhao.cx/go/src/runtime/time.go:102 +0x166 vendor/github.com/vinllen/mgo.(*mongoServer).pinger(0xc42026a5a0, 0xc42039c301) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/server.go:426 +0x4b3 created by vendor/github.com/vinllen/mgo.newServer /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/server.go:107 +0x186

goroutine 110 [chan receive]: mongoshake/collector/docsyncer.(DocExecutor).start(0xc42020a8d0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:148 +0x95 created by mongoshake/collector/docsyncer.(CollectionExecutor).Start /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:75 +0x161

goroutine 111 [chan receive]: mongoshake/collector/docsyncer.(DocExecutor).start(0xc42020a900) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:148 +0x95 created by mongoshake/collector/docsyncer.(CollectionExecutor).Start /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:75 +0x161

goroutine 192 [IO wait]: internal/poll.runtime_pollWait(0x7f334a2bf2d0, 0x72, 0xc420302c88) /home/zhuzhao.cx/go/src/runtime/netpoll.go:173 +0x57 internal/poll.(pollDesc).wait(0xc42023ab18, 0x72, 0xffffffffffffff00, 0xe21b60, 0x139ddd0) /home/zhuzhao.cx/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b internal/poll.(pollDesc).waitRead(0xc42023ab18, 0xc42034e400, 0x10, 0x10) /home/zhuzhao.cx/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d internal/poll.(FD).Read(0xc42023ab00, 0xc42034e410, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/internal/poll/fd_unix.go:157 +0x17d net.(netFD).Read(0xc42023ab00, 0xc42034e410, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/net/fd_unix.go:202 +0x4f net.(conn).Read(0xc42017a0b8, 0xc42034e410, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/net/net.go:176 +0x6a vendor/github.com/vinllen/mgo.fill(0xe2a140, 0xc42017a0b8, 0xc42034e410, 0x10, 0x10, 0x0, 0x11) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:601 +0x53 vendor/github.com/vinllen/mgo.(mongoSocket).readLoop(0xc4204b65a0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:619 +0x122 created by vendor/github.com/vinllen/mgo.newSocket /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:211 +0x1f5

goroutine 112 [chan receive]: mongoshake/collector/docsyncer.(DocExecutor).start(0xc42020a930) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:148 +0x95 created by mongoshake/collector/docsyncer.(CollectionExecutor).Start /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:75 +0x161

goroutine 113 [chan receive]: mongoshake/collector/docsyncer.(DocExecutor).start(0xc42020a960) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:148 +0x95 created by mongoshake/collector/docsyncer.(CollectionExecutor).Start /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:75 +0x161

goroutine 290 [chan receive]: mongoshake/collector/docsyncer.(DocExecutor).start(0xc42020a990) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:148 +0x95 created by mongoshake/collector/docsyncer.(CollectionExecutor).Start /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:75 +0x161

goroutine 291 [chan receive]: mongoshake/collector/docsyncer.(DocExecutor).start(0xc42020a9c0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:148 +0x95 created by mongoshake/collector/docsyncer.(CollectionExecutor).Start /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:75 +0x161

goroutine 292 [chan receive]: mongoshake/collector/docsyncer.(DocExecutor).start(0xc42020a9f0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:148 +0x95 created by mongoshake/collector/docsyncer.(CollectionExecutor).Start /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_executor.go:75 +0x161

goroutine 293 [select]: vendor/github.com/vinllen/mgo.(*mongoCluster).syncServersLoop(0xc420146a20) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/cluster.go:431 +0x31a created by vendor/github.com/vinllen/mgo.newCluster /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/cluster.go:83 +0x1ae

goroutine 259 [IO wait]: internal/poll.runtime_pollWait(0x7f334a2bf200, 0x72, 0xc42057ac88) /home/zhuzhao.cx/go/src/runtime/netpoll.go:173 +0x57 internal/poll.(pollDesc).wait(0xc42043e498, 0x72, 0xffffffffffffff00, 0xe21b60, 0x139ddd0) /home/zhuzhao.cx/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b internal/poll.(pollDesc).waitRead(0xc42043e498, 0xc42034e400, 0x10, 0x10) /home/zhuzhao.cx/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d internal/poll.(FD).Read(0xc42043e480, 0xc42034e420, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/internal/poll/fd_unix.go:157 +0x17d net.(netFD).Read(0xc42043e480, 0xc42034e420, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/net/fd_unix.go:202 +0x4f net.(conn).Read(0xc420454040, 0xc42034e420, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/net/net.go:176 +0x6a vendor/github.com/vinllen/mgo.fill(0xe2a140, 0xc420454040, 0xc42034e420, 0x10, 0x10, 0x0, 0x11) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:601 +0x53 vendor/github.com/vinllen/mgo.(mongoSocket).readLoop(0xc420268b40) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:619 +0x122 created by vendor/github.com/vinllen/mgo.newSocket /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:211 +0x1f5

goroutine 305 [sleep]: time.Sleep(0x37e11d600) /home/zhuzhao.cx/go/src/runtime/time.go:102 +0x166 vendor/github.com/vinllen/mgo.(*mongoServer).pinger(0xc4202f4b40, 0xc420212a01) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/server.go:426 +0x4b3 created by vendor/github.com/vinllen/mgo.newServer /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/server.go:107 +0x186

goroutine 295 [sleep]: time.Sleep(0x37e11d600) /home/zhuzhao.cx/go/src/runtime/time.go:102 +0x166 vendor/github.com/vinllen/mgo.(*mongoServer).pinger(0xc4202f4960, 0xc420212701) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/server.go:426 +0x4b3 created by vendor/github.com/vinllen/mgo.newServer /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/server.go:107 +0x186

goroutine 254 [IO wait]: internal/poll.runtime_pollWait(0x7f334a2bf130, 0x72, 0xc4204c1c88) /home/zhuzhao.cx/go/src/runtime/netpoll.go:173 +0x57 internal/poll.(pollDesc).wait(0xc420216618, 0x72, 0xffffffffffffff00, 0xe21b60, 0x139ddd0) /home/zhuzhao.cx/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b internal/poll.(pollDesc).waitRead(0xc420216618, 0xc42025a300, 0x10, 0x10) /home/zhuzhao.cx/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d internal/poll.(FD).Read(0xc420216600, 0xc42025a380, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/internal/poll/fd_unix.go:157 +0x17d net.(netFD).Read(0xc420216600, 0xc42025a380, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/net/fd_unix.go:202 +0x4f net.(conn).Read(0xc4202ac160, 0xc42025a380, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/net/net.go:176 +0x6a vendor/github.com/vinllen/mgo.fill(0xe2a140, 0xc4202ac160, 0xc42025a380, 0x10, 0x10, 0x0, 0x11) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:601 +0x53 vendor/github.com/vinllen/mgo.(mongoSocket).readLoop(0xc4200e3e60) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:619 +0x122 created by vendor/github.com/vinllen/mgo.newSocket /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:211 +0x1f5

goroutine 260 [IO wait]: internal/poll.runtime_pollWait(0x7f334a2bf060, 0x72, 0xc4204bbc88) /home/zhuzhao.cx/go/src/runtime/netpoll.go:173 +0x57 internal/poll.(pollDesc).wait(0xc4204a8118, 0x72, 0xffffffffffffff00, 0xe21b60, 0x139ddd0) /home/zhuzhao.cx/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b internal/poll.(pollDesc).waitRead(0xc4204a8118, 0xc420218700, 0x10, 0x10) /home/zhuzhao.cx/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d internal/poll.(FD).Read(0xc4204a8100, 0xc420218750, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/internal/poll/fd_unix.go:157 +0x17d net.(netFD).Read(0xc4204a8100, 0xc420218750, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/net/fd_unix.go:202 +0x4f net.(conn).Read(0xc420454050, 0xc420218750, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/net/net.go:176 +0x6a vendor/github.com/vinllen/mgo.fill(0xe2a140, 0xc420454050, 0xc420218750, 0x10, 0x10, 0x0, 0x1c) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:601 +0x53 vendor/github.com/vinllen/mgo.(mongoSocket).readLoop(0xc4202690e0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:619 +0x122 created by vendor/github.com/vinllen/mgo.newSocket /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:211 +0x1f5

goroutine 261 [IO wait]: internal/poll.runtime_pollWait(0x7f334a200070, 0x72, 0xc42050dc88) /home/zhuzhao.cx/go/src/runtime/netpoll.go:173 +0x57 internal/poll.(pollDesc).wait(0xc4200fa598, 0x72, 0xffffffffffffff00, 0xe21b60, 0x139ddd0) /home/zhuzhao.cx/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b internal/poll.(pollDesc).waitRead(0xc4200fa598, 0xc420218800, 0x10, 0x10) /home/zhuzhao.cx/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d internal/poll.(FD).Read(0xc4200fa580, 0xc4202188d0, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/internal/poll/fd_unix.go:157 +0x17d net.(netFD).Read(0xc4200fa580, 0xc4202188d0, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/net/fd_unix.go:202 +0x4f net.(conn).Read(0xc420454060, 0xc4202188d0, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/net/net.go:176 +0x6a vendor/github.com/vinllen/mgo.fill(0xe2a140, 0xc420454060, 0xc4202188d0, 0x10, 0x10, 0x0, 0x1c) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:601 +0x53 vendor/github.com/vinllen/mgo.(mongoSocket).readLoop(0xc420269680) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:619 +0x122 created by vendor/github.com/vinllen/mgo.newSocket /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:211 +0x1f5

goroutine 193 [IO wait]: internal/poll.runtime_pollWait(0x7f334a1fffa0, 0x72, 0xc4205c1c88) /home/zhuzhao.cx/go/src/runtime/netpoll.go:173 +0x57 internal/poll.(pollDesc).wait(0xc4203c0198, 0x72, 0xffffffffffffff00, 0xe21b60, 0x139ddd0) /home/zhuzhao.cx/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b internal/poll.(pollDesc).waitRead(0xc4203c0198, 0xc420540400, 0x10, 0x10) /home/zhuzhao.cx/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d internal/poll.(FD).Read(0xc4203c0180, 0xc420540400, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/internal/poll/fd_unix.go:157 +0x17d net.(netFD).Read(0xc4203c0180, 0xc420540400, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/net/fd_unix.go:202 +0x4f net.(conn).Read(0xc42017a0d0, 0xc420540400, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/net/net.go:176 +0x6a vendor/github.com/vinllen/mgo.fill(0xe2a140, 0xc42017a0d0, 0xc420540400, 0x10, 0x10, 0x0, 0x1c) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:601 +0x53 vendor/github.com/vinllen/mgo.(mongoSocket).readLoop(0xc4204b6a20) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:619 +0x122 created by vendor/github.com/vinllen/mgo.newSocket /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:211 +0x1f5

goroutine 218 [sleep]: time.Sleep(0x37e11d600) /home/zhuzhao.cx/go/src/runtime/time.go:102 +0x166 vendor/github.com/vinllen/mgo.(*mongoServer).pinger(0xc42025ed20, 0xc42023c301) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/server.go:426 +0x4b3 created by vendor/github.com/vinllen/mgo.newServer /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/server.go:107 +0x186

goroutine 310 [IO wait]: internal/poll.runtime_pollWait(0x7f334a1ffed0, 0x72, 0xc4205bdc88) /home/zhuzhao.cx/go/src/runtime/netpoll.go:173 +0x57 internal/poll.(pollDesc).wait(0xc4200fa798, 0x72, 0xffffffffffffff00, 0xe21b60, 0x139ddd0) /home/zhuzhao.cx/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b internal/poll.(pollDesc).waitRead(0xc4200fa798, 0xc42021e000, 0x10, 0x10) /home/zhuzhao.cx/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d internal/poll.(FD).Read(0xc4200fa780, 0xc42021e000, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/internal/poll/fd_unix.go:157 +0x17d net.(netFD).Read(0xc4200fa780, 0xc42021e000, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/net/fd_unix.go:202 +0x4f net.(conn).Read(0xc4203a4068, 0xc42021e000, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/net/net.go:176 +0x6a vendor/github.com/vinllen/mgo.fill(0xe2a140, 0xc4203a4068, 0xc42021e000, 0x10, 0x10, 0x0, 0x1c) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:601 +0x53 vendor/github.com/vinllen/mgo.(mongoSocket).readLoop(0xc4203dea20) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:619 +0x122 created by vendor/github.com/vinllen/mgo.newSocket /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:211 +0x1f5

goroutine 223 [IO wait]: internal/poll.runtime_pollWait(0x7f334a1ffe00, 0x72, 0xc42057bc88) /home/zhuzhao.cx/go/src/runtime/netpoll.go:173 +0x57 internal/poll.(pollDesc).wait(0xc420216718, 0x72, 0xffffffffffffff00, 0xe21b60, 0x139ddd0) /home/zhuzhao.cx/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b internal/poll.(pollDesc).waitRead(0xc420216718, 0xc4203a8b00, 0x10, 0x10) /home/zhuzhao.cx/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d internal/poll.(FD).Read(0xc420216700, 0xc4203a8bd0, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/internal/poll/fd_unix.go:157 +0x17d net.(netFD).Read(0xc420216700, 0xc4203a8bd0, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/net/fd_unix.go:202 +0x4f net.(conn).Read(0xc4202100e0, 0xc4203a8bd0, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/net/net.go:176 +0x6a vendor/github.com/vinllen/mgo.fill(0xe2a140, 0xc4202100e0, 0xc4203a8bd0, 0x10, 0x10, 0x0, 0x1c) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:601 +0x53 vendor/github.com/vinllen/mgo.(mongoSocket).readLoop(0xc4202cad80) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:619 +0x122 created by vendor/github.com/vinllen/mgo.newSocket /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:211 +0x1f5

goroutine 224 [IO wait]: internal/poll.runtime_pollWait(0x7f334a1ffd30, 0x72, 0xc4204bac88) /home/zhuzhao.cx/go/src/runtime/netpoll.go:173 +0x57 internal/poll.(pollDesc).wait(0xc420216818, 0x72, 0xffffffffffffff00, 0xe21b60, 0x139ddd0) /home/zhuzhao.cx/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b internal/poll.(pollDesc).waitRead(0xc420216818, 0xc42034e500, 0x10, 0x10) /home/zhuzhao.cx/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d internal/poll.(FD).Read(0xc420216800, 0xc42034e570, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/internal/poll/fd_unix.go:157 +0x17d net.(netFD).Read(0xc420216800, 0xc42034e570, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/net/fd_unix.go:202 +0x4f net.(conn).Read(0xc4202100f0, 0xc42034e570, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/net/net.go:176 +0x6a vendor/github.com/vinllen/mgo.fill(0xe2a140, 0xc4202100f0, 0xc42034e570, 0x10, 0x10, 0x0, 0x1c) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:601 +0x53 vendor/github.com/vinllen/mgo.(mongoSocket).readLoop(0xc4202cb320) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:619 +0x122 created by vendor/github.com/vinllen/mgo.newSocket /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:211 +0x1f5

goroutine 329 [select]: vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology.(Server).update(0xc4202840b0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology/server.go:420 +0x4f6 created by vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology.(Server).Connect /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology/server.go:175 +0x121

goroutine 358 [select]: vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology.(Topology).selectServerFromSubscription(0xc4205da0a0, 0xe26a80, 0xc420024088, 0xc4205ee1e0, 0xe21180, 0xc4205c84e0, 0xc4205ee180, 0xe21180, 0xc4205c84e0, 0xc4205ee180, ...) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology/topology.go:437 +0x167 vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology.(Topology).SelectServer(0xc4205da0a0, 0xe26a80, 0xc420024088, 0xe21180, 0xc4205c84e0, 0x0, 0x0, 0x0, 0x0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology/topology.go:334 +0x109 vendor/go.mongodb.org/mongo-driver/x/mongo/driver.Operation.selectServer(0xc4205b23e0, 0xcd75e6, 0x5, 0xe25f00, 0xc4205da0a0, 0xc4205b23f0, 0xe21180, 0xc4205c84e0, 0x0, 0x0, ...) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/operation.go:213 +0x104 vendor/go.mongodb.org/mongo-driver/x/mongo/driver.Operation.Execute(0xc4205b23e0, 0xcd75e6, 0x5, 0xe25f00, 0xc4205da0a0, 0xc4205b23f0, 0xe21180, 0xc4205c84e0, 0x0, 0x0, ...) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/operation.go:241 +0x176 vendor/go.mongodb.org/mongo-driver/x/mongo/driver/operation.(Command).Execute(0xc42015e3c0, 0xe26a80, 0xc420024088, 0xbec1e0, 0xc4205c8420) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/operation/command.go:75 +0x1f1 vendor/go.mongodb.org/mongo-driver/mongo.(Database).RunCommand(0xc4205ee120, 0xe26a80, 0xc420024088, 0xbec1e0, 0xc4205c8420, 0xc42059da90, 0x1, 0x1, 0x0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/mongo/database.go:191 +0x1ad vendor/go.mongodb.org/mongo-driver/mongo.(Client).Ping(0xc420102000, 0xe26a80, 0xc420024088, 0xc4205ba180, 0x0, 0x0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/mongo/client.go:232 +0x1c6 mongoshake/common.NewMongoCommunityConn(0xc42023606d, 0x1d, 0xc42023c355, 0x7, 0x1, 0xcd798d, 0x5, 0x0, 0x0, 0x458030, ...) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/common/community_client.go:69 +0x676 mongoshake/collector/docsyncer.(DocumentReader).ensureNetwork(0xc4205ee000, 0x9c, 0x3635312e) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_reader.go:301 +0x581 mongoshake/collector/docsyncer.(DocumentReader).NextDoc(0xc4205ee000, 0x0, 0x80, 0xc420524400) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_reader.go:272 +0x2f mongoshake/collector/docsyncer.(DBSyncer).splitSync(0xc4202d88c0, 0xc4205ee000, 0xc42023a480, 0xc42020cee0, 0x3733312e, 0x8a) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_syncer.go:455 +0xa3 mongoshake/collector/docsyncer.(DBSyncer).collectionSync.func1(0xc420512480, 0xc4202d88c0, 0xc42023a480, 0xc42020cee0, 0xc42034e5d0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_syncer.go:421 +0x8f created by mongoshake/collector/docsyncer.(DBSyncer).collectionSync /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_syncer.go:414 +0x2cc

goroutine 374 [select]: vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology.(Topology).selectServerFromSubscription(0xc4200d8280, 0xe26a80, 0xc420024088, 0xc42015aae0, 0xe21180, 0xc42062a140, 0xc42015aa80, 0xe21180, 0xc42062a140, 0xc42015aa80, ...) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology/topology.go:437 +0x167 vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology.(Topology).SelectServer(0xc4200d8280, 0xe26a80, 0xc420024088, 0xe21180, 0xc42062a140, 0x0, 0x0, 0x0, 0x0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology/topology.go:334 +0x109 vendor/go.mongodb.org/mongo-driver/x/mongo/driver.Operation.selectServer(0xc42020efa0, 0xcd75e6, 0x5, 0xe25f00, 0xc4200d8280, 0xc42020efb0, 0xe21180, 0xc42062a140, 0x0, 0x0, ...) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/operation.go:213 +0x104 vendor/go.mongodb.org/mongo-driver/x/mongo/driver.Operation.Execute(0xc42020efa0, 0xcd75e6, 0x5, 0xe25f00, 0xc4200d8280, 0xc42020efb0, 0xe21180, 0xc42062a140, 0x0, 0x0, ...) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/operation.go:241 +0x176 vendor/go.mongodb.org/mongo-driver/x/mongo/driver/operation.(Command).Execute(0xc420632000, 0xe26a80, 0xc420024088, 0xbec1e0, 0xc42062a0a0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/operation/command.go:75 +0x1f1 vendor/go.mongodb.org/mongo-driver/mongo.(Database).RunCommand(0xc42015aa20, 0xe26a80, 0xc420024088, 0xbec1e0, 0xc42062a0a0, 0xc420427a90, 0x1, 0x1, 0x0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/mongo/database.go:191 +0x1ad vendor/go.mongodb.org/mongo-driver/mongo.(Client).Ping(0xc42042e700, 0xe26a80, 0xc420024088, 0xc42034ad80, 0x0, 0x0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/mongo/client.go:232 +0x1c6 mongoshake/common.NewMongoCommunityConn(0xc42023606d, 0x1d, 0xc42023c355, 0x7, 0x1, 0xcd798d, 0x5, 0x0, 0x0, 0x458030, ...) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/common/community_client.go:69 +0x676 mongoshake/collector/docsyncer.(DocumentReader).ensureNetwork(0xc42015a900, 0x0, 0x0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_reader.go:301 +0x581 mongoshake/collector/docsyncer.(DocumentReader).NextDoc(0xc42015a900, 0x0, 0x80, 0xc4201f4800) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_reader.go:272 +0x2f mongoshake/collector/docsyncer.(DBSyncer).splitSync(0xc4202d88c0, 0xc42015a900, 0xc4200fa300, 0xc42020cf80, 0x0, 0x0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_syncer.go:455 +0xa3 mongoshake/collector/docsyncer.(DBSyncer).collectionSync.func1(0xc42049e060, 0xc4202d88c0, 0xc4200fa300, 0xc42020cf80, 0xc420214f10) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_syncer.go:421 +0x8f created by mongoshake/collector/docsyncer.(DBSyncer).collectionSync /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_syncer.go:414 +0x2cc

goroutine 281 [select]: vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology.(Topology).selectServerFromSubscription(0xc4202f8140, 0xe26a80, 0xc420024088, 0xc4206e8060, 0xe21180, 0xc4204c8660, 0xc4206e8000, 0xe21180, 0xc4204c8660, 0xc4206e8000, ...) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology/topology.go:437 +0x167 vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology.(Topology).SelectServer(0xc4202f8140, 0xe26a80, 0xc420024088, 0xe21180, 0xc4204c8660, 0x0, 0x0, 0x0, 0x0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology/topology.go:334 +0x109 vendor/go.mongodb.org/mongo-driver/x/mongo/driver.Operation.selectServer(0xc4206d4000, 0xcd75e6, 0x5, 0xe25f00, 0xc4202f8140, 0xc4206d4010, 0xe21180, 0xc4204c8660, 0x0, 0x0, ...) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/operation.go:213 +0x104 vendor/go.mongodb.org/mongo-driver/x/mongo/driver.Operation.Execute(0xc4206d4000, 0xcd75e6, 0x5, 0xe25f00, 0xc4202f8140, 0xc4206d4010, 0xe21180, 0xc4204c8660, 0x0, 0x0, ...) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/operation.go:241 +0x176 vendor/go.mongodb.org/mongo-driver/x/mongo/driver/operation.(Command).Execute(0xc420650000, 0xe26a80, 0xc420024088, 0xbec1e0, 0xc4204c85c0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/operation/command.go:75 +0x1f1 vendor/go.mongodb.org/mongo-driver/mongo.(Database).RunCommand(0xc42016f0e0, 0xe26a80, 0xc420024088, 0xbec1e0, 0xc4204c85c0, 0xc42041da90, 0x1, 0x1, 0x0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/mongo/database.go:191 +0x1ad vendor/go.mongodb.org/mongo-driver/mongo.(Client).Ping(0xc4201b6380, 0xe26a80, 0xc420024088, 0xc420200c60, 0x0, 0x0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/mongo/client.go:232 +0x1c6 mongoshake/common.NewMongoCommunityConn(0xc42023606d, 0x1d, 0xc42023c355, 0x7, 0x1, 0xcd798d, 0x5, 0x0, 0x0, 0x458030, ...) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/common/community_client.go:69 +0x676 mongoshake/collector/docsyncer.(DocumentReader).ensureNetwork(0xc42016ef60, 0xc4205615e8, 0x413f29) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_reader.go:301 +0x581 mongoshake/collector/docsyncer.(DocumentReader).NextDoc(0xc42016ef60, 0x0, 0x80, 0xc420358800) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_reader.go:272 +0x2f mongoshake/collector/docsyncer.(DBSyncer).splitSync(0xc4202d88c0, 0xc42016ef60, 0xc420216280, 0xc42020cf20, 0x0, 0x0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_syncer.go:455 +0xa3 mongoshake/collector/docsyncer.(DBSyncer).collectionSync.func1(0xc420512600, 0xc4202d88c0, 0xc420216280, 0xc42020cf20, 0xc4202a65a0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_syncer.go:421 +0x8f created by mongoshake/collector/docsyncer.(DBSyncer).collectionSync /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_syncer.go:414 +0x2cc

goroutine 377 [select]: vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology.(Server).update(0xc42068e000) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology/server.go:420 +0x4f6 created by vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology.(Server).Connect /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology/server.go:175 +0x121

goroutine 315 [select]: vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology.(Topology).selectServerFromSubscription(0xc4202f3400, 0xe26a80, 0xc420024088, 0xc4203dc7e0, 0xe21180, 0xc42020dba0, 0xc4202c9d40, 0xe21180, 0xc42020dba0, 0xc4202c9d40, ...) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology/topology.go:437 +0x167 vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology.(Topology).SelectServer(0xc4202f3400, 0xe26a80, 0xc420024088, 0xe21180, 0xc42020dba0, 0x0, 0x0, 0x0, 0x0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology/topology.go:334 +0x109 vendor/go.mongodb.org/mongo-driver/x/mongo/driver.Operation.selectServer(0xc4202a59b0, 0xcd75e6, 0x5, 0xe25f00, 0xc4202f3400, 0xc4202a59c0, 0xe21180, 0xc42020dba0, 0x0, 0x0, ...) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/operation.go:213 +0x104 vendor/go.mongodb.org/mongo-driver/x/mongo/driver.Operation.Execute(0xc4202a59b0, 0xcd75e6, 0x5, 0xe25f00, 0xc4202f3400, 0xc4202a59c0, 0xe21180, 0xc42020dba0, 0x0, 0x0, ...) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/operation.go:241 +0x176 vendor/go.mongodb.org/mongo-driver/x/mongo/driver/operation.(Command).Execute(0xc4206ba000, 0xe26a80, 0xc420024088, 0xbec1e0, 0xc42020dae0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/operation/command.go:75 +0x1f1 vendor/go.mongodb.org/mongo-driver/mongo.(Database).RunCommand(0xc4202c9ce0, 0xe26a80, 0xc420024088, 0xbec1e0, 0xc42020dae0, 0xc420429a90, 0x1, 0x1, 0x0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/mongo/database.go:191 +0x1ad vendor/go.mongodb.org/mongo-driver/mongo.(Client).Ping(0xc420652000, 0xe26a80, 0xc420024088, 0xc4202af560, 0x0, 0x0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/mongo/client.go:232 +0x1c6 mongoshake/common.NewMongoCommunityConn(0xc42023606d, 0x1d, 0xc42023c355, 0x7, 0x1, 0xcd798d, 0x5, 0x0, 0x0, 0x458030, ...) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/common/community_client.go:69 +0x676 mongoshake/collector/docsyncer.(DocumentReader).ensureNetwork(0xc4202c9bc0, 0x0, 0x0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_reader.go:301 +0x581 mongoshake/collector/docsyncer.(DocumentReader).NextDoc(0xc4202c9bc0, 0x0, 0x80, 0xc420640000) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_reader.go:272 +0x2f mongoshake/collector/docsyncer.(DBSyncer).splitSync(0xc4202d88c0, 0xc4202c9bc0, 0xc4202e4400, 0xc42020cf40, 0x0, 0x0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_syncer.go:455 +0xa3 mongoshake/collector/docsyncer.(DBSyncer).collectionSync.func1(0xc42016eb40, 0xc4202d88c0, 0xc4202e4400, 0xc42020cf40, 0xc420540600) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_syncer.go:421 +0x8f created by mongoshake/collector/docsyncer.(DBSyncer).collectionSync /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_syncer.go:414 +0x2cc

goroutine 435 [IO wait]: internal/poll.runtime_pollWait(0x7f334a1ff2a0, 0x72, 0xc42029ac88) /home/zhuzhao.cx/go/src/runtime/netpoll.go:173 +0x57 internal/poll.(pollDesc).wait(0xc4204a8398, 0x72, 0xffffffffffffff00, 0xe21b60, 0x139ddd0) /home/zhuzhao.cx/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b internal/poll.(pollDesc).waitRead(0xc4204a8398, 0xc42069c500, 0x10, 0x10) /home/zhuzhao.cx/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d internal/poll.(FD).Read(0xc4204a8380, 0xc42069c510, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/internal/poll/fd_unix.go:157 +0x17d net.(netFD).Read(0xc4204a8380, 0xc42069c510, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/net/fd_unix.go:202 +0x4f net.(conn).Read(0xc4206f8040, 0xc42069c510, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/net/net.go:176 +0x6a vendor/github.com/vinllen/mgo.fill(0xe2a140, 0xc4206f8040, 0xc42069c510, 0x10, 0x10, 0x0, 0x11) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:601 +0x53 vendor/github.com/vinllen/mgo.(mongoSocket).readLoop(0xc4200e2480) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:619 +0x122 created by vendor/github.com/vinllen/mgo.newSocket /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:211 +0x1f5

goroutine 320 [select]: vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology.(Topology).selectServerFromSubscription(0xc420692000, 0xe26a80, 0xc420024088, 0xc4203dc8a0, 0xe21180, 0xc4203a0c60, 0xc4203dc660, 0xe21180, 0xc4203a0c60, 0xc4203dc660, ...) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology/topology.go:437 +0x167 vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology.(Topology).SelectServer(0xc420692000, 0xe26a80, 0xc420024088, 0xe21180, 0xc4203a0c60, 0x0, 0x0, 0x0, 0x0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology/topology.go:334 +0x109 vendor/go.mongodb.org/mongo-driver/x/mongo/driver.Operation.selectServer(0xc420508bb0, 0xcd75e6, 0x5, 0xe25f00, 0xc420692000, 0xc420508bc0, 0xe21180, 0xc4203a0c60, 0x0, 0x0, ...) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/operation.go:213 +0x104 vendor/go.mongodb.org/mongo-driver/x/mongo/driver.Operation.Execute(0xc420508bb0, 0xcd75e6, 0x5, 0xe25f00, 0xc420692000, 0xc420508bc0, 0xe21180, 0xc4203a0c60, 0x0, 0x0, ...) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/operation.go:241 +0x176 vendor/go.mongodb.org/mongo-driver/x/mongo/driver/operation.(Command).Execute(0xc4206ac000, 0xe26a80, 0xc420024088, 0xbec1e0, 0xc4203a0bc0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/operation/command.go:75 +0x1f1 vendor/go.mongodb.org/mongo-driver/mongo.(Database).RunCommand(0xc4203dc600, 0xe26a80, 0xc420024088, 0xbec1e0, 0xc4203a0bc0, 0xc420425a90, 0x1, 0x1, 0x0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/mongo/database.go:191 +0x1ad vendor/go.mongodb.org/mongo-driver/mongo.(Client).Ping(0xc42068c000, 0xe26a80, 0xc420024088, 0xc420542870, 0x0, 0x0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/mongo/client.go:232 +0x1c6 mongoshake/common.NewMongoCommunityConn(0xc42023606d, 0x1d, 0xc42023c355, 0x7, 0xc4205d2c01, 0xcd798d, 0x5, 0x0, 0x0, 0x7f334a1abce0, ...) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/common/community_client.go:69 +0x676 mongoshake/collector/docsyncer.(DocumentReader).ensureNetwork(0xc4203dc4e0, 0x0, 0x0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_reader.go:301 +0x581 mongoshake/collector/docsyncer.(DocumentReader).NextDoc(0xc4203dc4e0, 0x0, 0x80, 0xc4202b6c00) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_reader.go:272 +0x2f mongoshake/collector/docsyncer.(DBSyncer).splitSync(0xc4202d88c0, 0xc4203dc4e0, 0xc42023a600, 0xc42020cf00, 0x0, 0x0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_syncer.go:455 +0xa3 mongoshake/collector/docsyncer.(DBSyncer).collectionSync.func1(0xc4202c9440, 0xc4202d88c0, 0xc42023a600, 0xc42020cf00, 0xc420540640) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_syncer.go:421 +0x8f created by mongoshake/collector/docsyncer.(DBSyncer).collectionSync /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_syncer.go:414 +0x2cc

goroutine 282 [select]: vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology.(Server).update(0xc420642000) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology/server.go:420 +0x4f6 created by vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology.(Server).Connect /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology/server.go:175 +0x121

goroutine 321 [select]: vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology.(Server).update(0xc42069a000) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology/server.go:420 +0x4f6 created by vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology.(Server).Connect /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology/server.go:175 +0x121

goroutine 334 [select]: vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology.(Topology).selectServerFromSubscription(0xc4206ea0a0, 0xe26a80, 0xc420024088, 0xc4205ee660, 0xe21180, 0xc4206e2280, 0xc4205ee600, 0xe21180, 0xc4206e2280, 0xc4205ee600, ...) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology/topology.go:437 +0x167 vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology.(Topology).SelectServer(0xc4206ea0a0, 0xe26a80, 0xc420024088, 0xe21180, 0xc4206e2280, 0x0, 0x0, 0x0, 0x0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology/topology.go:334 +0x109 vendor/go.mongodb.org/mongo-driver/x/mongo/driver.Operation.selectServer(0xc4206c8190, 0xcd75e6, 0x5, 0xe25f00, 0xc4206ea0a0, 0xc4206c81a0, 0xe21180, 0xc4206e2280, 0x0, 0x0, ...) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/operation.go:213 +0x104 vendor/go.mongodb.org/mongo-driver/x/mongo/driver.Operation.Execute(0xc4206c8190, 0xcd75e6, 0x5, 0xe25f00, 0xc4206ea0a0, 0xc4206c81a0, 0xe21180, 0xc4206e2280, 0x0, 0x0, ...) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/operation.go:241 +0x176 vendor/go.mongodb.org/mongo-driver/x/mongo/driver/operation.(Command).Execute(0xc42015e5a0, 0xe26a80, 0xc420024088, 0xbec1e0, 0xc4206e2200) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/operation/command.go:75 +0x1f1 vendor/go.mongodb.org/mongo-driver/mongo.(Database).RunCommand(0xc4205ee5a0, 0xe26a80, 0xc420024088, 0xbec1e0, 0xc4206e2200, 0xc42059ba90, 0x1, 0x1, 0x0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/mongo/database.go:191 +0x1ad vendor/go.mongodb.org/mongo-driver/mongo.(Client).Ping(0xc4200fe380, 0xe26a80, 0xc420024088, 0xc420716030, 0x0, 0x0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/mongo/client.go:232 +0x1c6 mongoshake/common.NewMongoCommunityConn(0xc42023606d, 0x1d, 0xc42023c355, 0x7, 0xc42056bc01, 0xcd798d, 0x5, 0x0, 0x0, 0x7f334a19b010, ...) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/common/community_client.go:69 +0x676 mongoshake/collector/docsyncer.(DocumentReader).ensureNetwork(0xc42049e960, 0xc42056bde8, 0x41413d) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_reader.go:301 +0x581 mongoshake/collector/docsyncer.(DocumentReader).NextDoc(0xc42049e960, 0x0, 0x80, 0xc42066c400) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_reader.go:272 +0x2f mongoshake/collector/docsyncer.(DBSyncer).splitSync(0xc4202d88c0, 0xc42049e960, 0xc42043e080, 0xc42020cf60, 0x0, 0x0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_syncer.go:455 +0xa3 mongoshake/collector/docsyncer.(DBSyncer).collectionSync.func1(0xc42009ca20, 0xc4202d88c0, 0xc42043e080, 0xc42020cf60, 0xc42021e390) /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_syncer.go:421 +0x8f created by mongoshake/collector/docsyncer.(DBSyncer).collectionSync /home/zhuzhao.cx/mongo-shake/MongoShake/src/mongoshake/collector/docsyncer/doc_syncer.go:414 +0x2cc

goroutine 380 [select]: vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology.(Server).update(0xc4206d80b0) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology/server.go:420 +0x4f6 created by vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology.(Server).Connect /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/go.mongodb.org/mongo-driver/x/mongo/driver/topology/server.go:175 +0x121

goroutine 340 [IO wait]: internal/poll.runtime_pollWait(0x7f334a1ff440, 0x72, 0xc420575c88) /home/zhuzhao.cx/go/src/runtime/netpoll.go:173 +0x57 internal/poll.(pollDesc).wait(0xc4204a8298, 0x72, 0xffffffffffffff00, 0xe21b60, 0x139ddd0) /home/zhuzhao.cx/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b internal/poll.(pollDesc).waitRead(0xc4204a8298, 0xc4206fa200, 0x10, 0x10) /home/zhuzhao.cx/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d internal/poll.(FD).Read(0xc4204a8280, 0xc4206fa230, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/internal/poll/fd_unix.go:157 +0x17d net.(netFD).Read(0xc4204a8280, 0xc4206fa230, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/net/fd_unix.go:202 +0x4f net.(conn).Read(0xc42000e008, 0xc4206fa230, 0x10, 0x10, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/net/net.go:176 +0x6a vendor/github.com/vinllen/mgo.fill(0xe2a140, 0xc42000e008, 0xc4206fa230, 0x10, 0x10, 0x0, 0x11) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:601 +0x53 vendor/github.com/vinllen/mgo.(mongoSocket).readLoop(0xc42032c120) /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:619 +0x122 created by vendor/github.com/vinllen/mgo.newSocket /home/zhuzhao.cx/mongo-shake/MongoShake/src/vendor/github.com/vinllen/mgo/socket.go:211 +0x1f5

goroutine 397 [IO wait]: internal/poll.runtime_pollWait(0x7f334a1ff1d0, 0x72, 0xc4203d3e58) /home/zhuzhao.cx/go/src/runtime/netpoll.go:173 +0x57 internal/poll.(pollDesc).wait(0xc42069e618, 0x72, 0xffffffffffffff00, 0xe21b60, 0x139ddd0) /home/zhuzhao.cx/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b internal/poll.(pollDesc).waitRead(0xc42069e618, 0xc4206e6700, 0x1, 0x1) /home/zhuzhao.cx/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d internal/poll.(FD).Read(0xc42069e600, 0xc4206e6731, 0x1, 0x1, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/internal/poll/fd_unix.go:157 +0x17d net.(netFD).Read(0xc42069e600, 0xc4206e6731, 0x1, 0x1, 0xc42049e480, 0x0, 0xc42031a758) /home/zhuzhao.cx/go/src/net/fd_unix.go:202 +0x4f net.(conn).Read(0xc420210118, 0xc4206e6731, 0x1, 0x1, 0x0, 0x0, 0x0) /home/zhuzhao.cx/go/src/net/net.go:176 +0x6a net/http.(connReader).backgroundRead(0xc4206e6720) /home/zhuzhao.cx/go/src/net/http/server.go:668 +0x5a created by net/http.(*connReader).startBackgroundRead /home/zhuzhao.cx/go/src/net/http/server.go:664 +0xce

49566258 commented 3 years ago

无意中把 sync_mode = full改为sync_mode = 空后发现可以同步了。。。。 2874077, 907]], list_unack [0] [2020/11/27 14:35:17 CST] [INFO] Replayer-3 Executor-3 doSync oplogRecords received[1024] merged[1024]. merge to 100.00% chunks [2020/11/27 14:35:17 CST] [INFO] Collector-worker-3 transfer retransmit:false send [1024] logs. reply_acked [6884291744616153399[1602874078, 311]], list_unack [0] [2020/11/27 14:35:17 CST] [INFO] Replayer-3 Executor-3 doSync oplogRecords received[1024] merged[1024]. merge to 100.00% chunks [2020/11/27 14:35:17 CST] [INFO] Collector-worker-3 transfer retransmit:false send [1024] logs. reply_acked [6884291744616154423[1602874078, 1335]], list_unack [0] [2020/11/27 14:35:17 CST] [INFO] [name=RS5, stage=incr, get=1527808, filter=0, success=1261852, tps=3072, ckpt_times=21, lsn_ckpt={6884291727436284654[1602874074, 750], 2020-10-17 02:47:54}, lsn_ack={6884291860580270740[1602874105, 660], 2020-10-17 02:48:25}]] [2020/11/27 14:35:17 CST] [INFO] Replayer-3 Executor-3 doSync oplogRecords received[1024] merged[1024]. merge to 100.00% chunks [2020/11/27 14:35:17 CST] [INFO] Collector-worker-3 transfer retransmit:false send [1024] logs. reply_acked [6884291748911121151[1602874079, 767]], list_unack [0] [2020/11/27 14:35:18 CST] [INFO] Replayer-3 Executor-3 doSync oplogRecords received[1024] merged[1024]. merge to 100.00% chunks [2020/11/27 14:35:18 CST] [INFO] Collector-worker-3 transfer retransmit:false send [1024] logs. reply_acked [6884291748911122175[1602874079, 1791]], list_unack [0] [2020/11/27 14:35:18 CST] [INFO] Replayer-3 Executor-3 doSync oplogRecords received[1024] merged[1024]. merge to 100.00% chunks [2020/11/27 14:35:18 CST] [INFO] Collector-worker-3 transfer retransmit:false send [1024] logs. reply_acked [6884291753206088593[1602874080, 913]], list_unack [0] [2020/11/27 14:35:18 CST] [INFO] Replayer-6 Executor-6 doSync oplogRecords received[10] merged[7]. merge to 70.00% chunks

vinllen commented 3 years ago

[2020/11/27 14:11:46 CST] [CRIT] splitter reader[DocumentReader src[mongodb://10.168.168.17:27002] ns[{db_news_rank t_news_ban}] query[map[]]] get next document failed: ping to mongodb://10.168.168.17:27002 failed: server selection error: server selection timeout, current topology: { Type: Unknown, Servers: [{ Addr: 10.168.168.17:27002, Type: RSSecondary, State: Connected, Average RTT: 535234 }, ] } 3.0以前的版本支持有问题，所以mongoshake要求最低版本是3.0。sync_mod留空默认是增量同步，全量被跳过了。

49566258 commented 3 years ago

谢谢了，也就是当前还不支持3。0以前版本了，现在看增量同步过来一些数据（目标库之前是空的，但一直跑不停，也不再增加新内容），看来得先把源库做升级了。。。

vinllen commented 3 years ago

你可以升级3.x版本再试一下。 当前问题先关了，有问题可以再建一个issue，相同问题可以重新打开。