Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications. Nomad is easy to operate and scale and has native Consul and Vault integrations.
As of version 1.5 Nomad uses a special TimeTable data structure to track approximate timestamps for Raft indexes. These timestamps are used by Nomad's state garbage collection to determine how old an object is based on its Raft index compared to the wallclock time and GC threshold for that particular object.
For example in the case of Nodes:
The default node_gc_threshold is 24 hours.
Node A went down at Raft index 123
The TimeTable recorded 2009-11-10 23:00:00 UTC as the Nearest Time for Raft index 123.
Therefore at the first Node GC internal after 2009-11-11 23:00:00 UTC, Node A is eligible for garbage collection.
Since all objects record their Raft create and/or modified index, TimeTable has historically provided a generic mechanism for determining when any object is created or modified. TimeTable must make a tradeoff between space used, history kept, and precision:
History
Precision
Entries
Size
Current
72h
5m
864
27kb
30 days / High Precision
720h
5m
8,640
270kb
30 days / Low Precision
720h
1h
720 entries
23kb
Thanks to being able to binary search (O(log N)) to find its time entry, TimeTable lookups are fairly cheap even for the 8.6k entry case.
However TimeTable is extra accounting (cpu), extra memory, extra state to snapshot. Never very much, but always overhead if there's a timestamp that can be used instead.
Proposal
Deprecate TimeTable and move all uses to object-based timestamps. The following objects and operations would need to be migrated:
tgross
commented
1 year ago
Background
As of version 1.5 Nomad uses a special TimeTable data structure to track approximate timestamps for Raft indexes. These timestamps are used by Nomad's state garbage collection to determine how old an object is based on its Raft index compared to the wallclock time and GC threshold for that particular object.
For example in the case of Nodes:
node_gc_thresholdis 24 hours.downat Raft index 1232009-11-10 23:00:00 UTCas the Nearest Time for Raft index 123.Therefore at the first Node GC internal after
2009-11-11 23:00:00 UTC, Node A is eligible for garbage collection.Since all objects record their Raft create and/or modified index, TimeTable has historically provided a generic mechanism for determining when any object is created or modified. TimeTable must make a tradeoff between space used, history kept, and precision:
Thanks to being able to binary search (
O(log N)) to find its time entry, TimeTable lookups are fairly cheap even for the 8.6k entry case.However TimeTable is extra accounting (cpu), extra memory, extra state to snapshot. Never very much, but always overhead if there's a timestamp that can be used instead.
Proposal
Deprecate TimeTable and move all uses to object-based timestamps. The following objects and operations would need to be migrated: