Migrate data from IPFS Cluster - Githubissues

web3-storage / backup

A tool to backup all data for disaster recovery.

0 stars 0 forks source link

Migrate data from IPFS Cluster #1

Closed alanshaw closed 1 year ago

alanshaw commented 1 year ago

We need to migrate data off of IPFS Cluster that is not already in buckets. Rough plan:

Determine list(s) of CIDs to migrate

Consider both upload.backup_urls as well as backup table when determining whether to backup
web3.storage also needs to consider psa_pin_request

Changes needed to this repo

Read an ndjson list of things to backup
Start from index (for restarts)
Replace updating DB with logging success failure
Output results