MatrixAI / Emergence

Distributed Infrastructure Orchestration
Apache License 2.0
1 stars 0 forks source link

Roll-back strategy and snapshot for each propagation changes. #56

Open jeannieyeliu opened 5 years ago

jeannieyeliu commented 5 years ago

Snapshot

The Graph Systems needs to record the changes during each Propagation recomputation procedure. This is mainly for the following purpose:

  1. user wants to roll-back to a certain stage
  2. recomputation gets a ridiculous value
  3. users wants to use certain configuration under different circumstances
CMCDragonkai commented 5 years ago

This is related to tree versioning. But the main idea is that one may make a change to the graph through an operator change to the Architect expressions. We need to talk about the difference between commands and expressions though, as commands is what performs side effects. Anyway, one a command is made, a rollback can be initiated to go back to any point in version from prior commands.

jeannieyeliu commented 5 years ago

tree versioning

Something similar to git version management?

nzhang-zh commented 5 years ago
  1. recomputation gets a ridiculous value

Invalid computation results should ideally not become a snapshot?

Maybe during graph computation, instead of mutating existing nodes, create new nodes and link to previous version of the same node. So that we have a trail of change history, and for multi layer/step computations, if a later layer/step fails, earlier computation results can simply be discarded.

nzhang-zh commented 5 years ago

Adding on to the purpose of snapshot, apart from user specifying different snapshots, Adaptation/Emergence could keep track of several snapshots as profiles and switch between them based on different metrics such as request per min, time of day, geographical location of request sources etc.

CMCDragonkai commented 5 years ago

I mean similar to MVCC, see persistent data structures.

jeannieyeliu commented 5 years ago

MVCC https://github.com/MatrixAI/js-tree-order-index

jeannieyeliu commented 5 years ago

Several ways of tree versioning:

  1. MVCC
  2. Historical entries
  3. purely functional data structure such as: fat node strategy iterator snapshot