(Pretty much copied from 56c6408, the other commits are just cleanups to make the "big one" easier to read as diff)
When scanning the oplog there are many entries that are not directly relevant for the current river: administrative messages, no-op entries, entries for other collections, ... .
Previously we simply ignored these.
This "ignoring" does have quite grave consequences:
When the river is restarted it may have to re-process many of them, especially when it is a for a collection with little activity, in a database with very active other collections.
As a direct consequence of that the amount of oplog entries that need to be available must be such that of every collection the newest eligible entry is still in the collection.
Especially the last point is problematic when the cluster topology itself is very dynamic, as it can happen in automatically managed clusters in a cloud environment. [see below]
This commit changes the approach so that for each bulk of oplog entries at least the timestamp in the river state is updated: either because there was at least one eligible entry, or, failing that, by introducing a "no-op" request into the index stream.
The problem I observed is that I started to let AWS' autoscaling manage our cluster of mongods, which means that the PRIMARY moves around quite often - with "move" meaning "instance gets completely replaced". Whenever such a move happened my rivers started becoming stale, and needed to be recreated.
This patch set fixes that for the common case of an orderly shutdown of mongod, because we no longer need to have access to old oplog entries as long as the river can follow the oplog closely, and write the timestamp information out quickly enough.
(Pretty much copied from 56c6408, the other commits are just cleanups to make the "big one" easier to read as diff)
When scanning the oplog there are many entries that are not directly relevant for the current river: administrative messages, no-op entries, entries for other collections, ... . Previously we simply ignored these.
This "ignoring" does have quite grave consequences:
Especially the last point is problematic when the cluster topology itself is very dynamic, as it can happen in automatically managed clusters in a cloud environment. [see below]
This commit changes the approach so that for each bulk of oplog entries at least the timestamp in the river state is updated: either because there was at least one eligible entry, or, failing that, by introducing a "no-op" request into the index stream.
The problem I observed is that I started to let AWS' autoscaling manage our cluster of
mongods, which means that the PRIMARY moves around quite often - with "move" meaning "instance gets completely replaced". Whenever such a move happened my rivers started becoming stale, and needed to be recreated.This patch set fixes that for the common case of an orderly shutdown of mongod, because we no longer need to have access to old oplog entries as long as the river can follow the oplog closely, and write the timestamp information out quickly enough.