NVIDIA / aistore

AIStore: scalable storage for AI applications
https://aistore.nvidia.com
MIT License
1.21k stars 160 forks source link

Migration of data between deployments #172

Closed satyatumati closed 5 months ago

satyatumati commented 6 months ago

Describe the issue

We have a kubernetes cluster of older aistore 3.11. We decided to deploy a new cluster with latest version of aistore. Is there a way to move the data between deployments?

Links

na

alex-aizman commented 6 months ago

multiple ways to do it but maybe the easiest would be - via backup to Cloud. Incidentally, backup is always a good idea - you are not supposed to have a single copy of any production data. Anyway, example:

$ ais bucket cp ais://src s3://dst

This then prints out a message on how to monitor the progress. Once the data is safely backed up, you can then get rid of the 3.11 without any fear. And then deploy v3.22 from scratch.

P.S. As aside, there'd be an interesting exercise to attach - as in: ais cluster remote-attach - 3.11 cluster to 3.22 and see if that works. And I can imagine several other things, including vanilla rsync. But checking out anything like that will take time that we simply don't have. Hence, Cloud.