Ease persisted Raft data schema modification

[notnoop]

When a struct is persisted in Raft, we must support its schema "forever". Consider Allocation.TaskResources field migration to Allocation.AllocatedResources.Task, we effectively cannot drop Allocation.TaskResources ever from the allocation struct. If an allocation is created in a Nomad 0.8 (pre AllocatedResources intro) cluster, and the cluster is upgraded in rapid secession to Nomad 0.9 then 0.10, ... then 0.N; at version 0.N we aren't assured that the persisted Raft representation AllocatedResources is ever populated, and thus always need to fallback to reading TaskResources fields.

We can guarantee that a field is no longer needed, if we guarantee that Raft Snapshot and Client Store data have been migrate to latest schema version

Next Steps:

• Document the backward compatibility requirement and approaches for persisted data structures for engineers
• Consider some approaches to force a snapshot during an upgrade process. Some ideas:
  • force taking snapshot after an upgrade; add an Force Snapshot api/cli endpoint to be called during an upgrade

[stale[bot]]

Hey there

Since this issue hasn't had any activity in a while - we're going to automatically close it in 30 days. If you're still seeing this issue with the latest version of Nomad, please respond here and we'll keep this open and take another look at this.

Thanks!