search

archiver-appliance / epicsarchiverap

This is an implementation of an archiver for EPICS control systems that aims to archive millions of PVs.
Other
38 stars 38 forks source link

How to expand the storage space of LTS？ #84

Closed yifans closed 5 years ago

yifans commented 5 years ago

We deployed AA on a rack server, which has been running for two years. Long-term data is stored on local disks (4T) and half of the space has been used.

How to expand the storage space of long-term data? At the same time, the expansion process ensures that the stored data are not lost.

Thanks.

slacmshankar commented 5 years ago

I know folks use LVM to configure their local file systems to be expandable. However, you may get a better answer from your IT dept; this aspect of the question is probably more suited to them.

I can however address the archiver related aspects. Some years ago we had to move LTS data from one NFS server to another. This is approximately the same problem as what you have and I can tell you what we did.

  1. Make backups of everything :).
  2. The ETL transfers (especially the MTS -> LTS) always happen at predictable times. You can look at the PV details for any of your high traffic PV's and determine when this happens. These are usually in the fields "ETL 1 last completed" and "ETL 1 next job runs at". You can then schedule this work to make sure any LTS disruption is minimal. Schedule this work when ETL is NOT transferring data to the LTS.
  3. We had to move data from NFS server NFS1 to NFS server NFS2. The principal tool we used was rsync. We did an initial rsync of data from NFS1 to NFS2; this took a couple of days as the size of the data was large.
  4. rsync can transfer files incrementally; so; you can run the same rsync command again, only the deltas will be copied. Since all the LTS files are append-only, running rsync again in your scheduled window will only transfer the deltas.
  5. We shutdown the archivers and ran rsync again to make sure all the LTS data is copied over to NFS2. We used checksums on a few of the files to make sure all the data was in sync.
  6. We use softlinks to control where the LTS is; we then updated the softlink to point to NFS2 and started the archivers.
  7. We tested over the next couple of days using retrieval for a few PV's to make sure all the data made it thru and that ETL was moving the data correctly.
  8. Finally, when we were satisfied, we removed the old NFS server NFS1.

This is not an exact answer to your question; but I hope that this helps.