Backup relay/proxy without copying bitcoind entire chain

stakwork / sphinx-swarm

lightning container orchestration for massive deployments

4 stars 4 forks source link

Open tomsmith8 opened 4 months ago

tomsmith8 commented 4 months ago

Backup relay and proxy volumes to S3.

use download_from_container to get and zip the entire root volume for proxy and relay
use s3 api https://crates.io/crates/rust-s3 to connect to an S3 bucket subfolder
name the zip file with the data/time
upload it to the S3 bucket
look through the bucket and delete old ones (based on name)

Evanfeenstra commented 4 months ago

docker API (in swarm) has the ability to take a snapshot of a single volume, so we could potentially snapshot relay+proxy every day. Also maybe LND.

Like zip it to a file and upload to S3

Evanfeenstra commented 3 months ago

Task

Incident which forced us to create an AWS snapshot. However, this means we have to copy the entire bitcoind chain which can take a lot of time.

We need to figure out a solution to backup relay/proxy without copying the entire chain