search

loomnetwork / loomchain

Loom DAppChain Engine
Other
166 stars 32 forks source link

Implement node rollback #618

Open enlight opened 5 years ago

enlight commented 5 years ago

Currently when a node fails due to an app hash mismatch we have to restore the node from a backup, this is very time consuming, and error prone. We need to build a tool to rollback the node state to a point before the app hash mismatch occurred (and try again from that point either with the same build or another build).

Depending on what caused the app hash mismatch in the first place it may be sufficient to rollback just the IAVL store (app.db), or it may be necessary to rollback the TM state and block stores.

The precursor for this tool already exists in the tm-tools repo, but the rollback functionality there was built for TM 0.22.8 node, so it needs to be repurposed for TM 0.27.X. The IAVL store rollback code already exists for TM 0.26.X, and probably supports TM 0.27.4 as is.

enlight commented 5 years ago

Relevant issue about rolling back state https://github.com/tendermint/tendermint/issues/3072

ParthDesai commented 5 years ago

Implemented here: https://github.com/loomnetwork/tm-tools/pull/7