All actions are prompted for confirmation before executing (with no bypass).
IPs are loaded via an optioned logs-directory (i.e: ./logs/s3/). However, the script will ask for confirmation on the loaded IPs.
The script will allow the user to choose a subset of loaded IPs (for each shard) and/or a subset of shard(s). Alternatively, a user can manually enter IPs for each shard as a CSV string of IPs.
The user can manually enter the s3 bucket path for the snapshot DB that they would like to recovery with (for each shard's DB), or they can interactively choose the DBs from a list of snapshots sorted in descending order by block.
The script will ALWAYS backup existing DB on ALL nodes (locally) before recovering from a snapshot.
The user can choose to restart all of the given/selected IPs after recovery & check for a running node with progress
The script will use some assumptions given by the nature of being in the pipeline. That said, the script makes such assumptions in contained units, therefore minimal edits would be needed should such assumpts be no longer valid. Here are the assumptions:
node_ssh.sh needs to be in the same directory as the script and never requires interaction.
/../tools/snapshot/rclone.conf contains the default rclone config for
a node to download the snapshot db.
This closes https://github.com/harmony-one/harmony/issues/2998
Recovery design
./logs/s3/). However, the script will ask for confirmation on the loaded IPs.The script will use some assumptions given by the nature of being in the
pipeline. That said, the script makes such assumptions in contained units, therefore minimal edits would be needed should such assumpts be no longer valid. Here are the assumptions:Internally here is the doc.