search

Percona-Lab / mongodb_consistent_backup

A tool for performing consistent backups of MongoDB Clusters or Replica Sets
https://www.percona.com
Apache License 2.0
276 stars 81 forks source link

Missing oplog entries? #258

Open gulikoza opened 6 years ago

gulikoza commented 6 years ago

Hi,

We are evaluating mongodb_consistent_backup. I'm trying to understand how the program stores oplog changes. We are using sharded replica set. I'm checking the logs for config replica backup and I'm seeing something like:

Done tailing oplog on configRepl/10.xx.xx.xx:27019, 16010 oplog changes, end ts: Timestamp(1523625054, 1)

But the last document in configRepl.tar/dump/oplog.bson has timestamp 1523625034, so it would appear oplog dump ends 20 secs earlier. I did a:

# bsondump --pretty oplog.bson |grep "timestamp"| wc -l
2018-04-14T21:33:32.343+0500    15988 objects found
15988

and it would appear it's missing around 22 entries.

I am reading the statistics correctly?

gulikoza commented 6 years ago

OK, so the Resolver trims the oplogs to max_ts. I see that. But this timestamp is not recorded in meta.json? How would you know which is the last ts?