PowerDNS / lightningstream

Lightning Stream syncs LMDB databases through S3 buckets between multiple servers, including PowerDNS Authoritative server 4.8+ LMDBs
https://doc.powerdns.com/lightningstream/
MIT License
28 stars 16 forks source link

docs: add section for S3 migration ? #61

Open udf2457 opened 1 year ago

udf2457 commented 1 year ago

Please consider enhancing the docs by adding a section where you describe the process for migrating between S3 stores.

For example, say you start on MinIO and you wish to migrate to AWS S3, or vice-versa or any other number of examples.

It is not immediately clear how this would be safely done ?

wojas commented 1 year ago

You can safely copy all snapshots from Minio, upload them to another store like AWS S3, and update the configuration to point at the new S3 backend. You may want to bring the API or server down to prevent changes during this migration. Perhaps we could add a lightningstream snapshots backup and restore command in the future to make this easier.

Alternatively, you could just point all existing instances at a new empty destination. Since their LMDBs contain all the data, this will write new snapshots that contain all the data needed. If you do a rolling update even the latest changes will get incorporated everywhere. This is probably the easiest way to migrate if you are not running ephemeral instances that start empty after a configuration change.

I agree that it is worth adding this to the documentation.

udf2457 commented 1 year ago

Thanks @wojas. Very useful