Is your feature request related to a problem? Please describe.
Data always goes into galvanalyser but never comes out..... Eventually you'll either run out of disk space or end up paying a lot for disks
Describe the solution you'd like
Functionality to archive a dataset to cold storage.
This storage would just be an S3 bucket, defined in .env (url, username, password)
Button in UI to initiate archive for a given dataset
data for that dataset is written to a hdf5 file (or other suitable, compressible binary format suitable for long term storage)
file is written to S3 bucket to a filename based on the dataset id
dataset is deleted from database, but the metadata is retained. Add flag to metadata indicating that this dataset has been archived
Button in UI to initiate loading from archive
data is read from file in S3 bucket back into the database
archive flag cleared
General: there needs to be checks to handle archived data (e.g. don't plot datasets that have been archived). API endpoints to provide data error if the dataset has been archived.
Is your feature request related to a problem? Please describe.
Data always goes into galvanalyser but never comes out..... Eventually you'll either run out of disk space or end up paying a lot for disks
Describe the solution you'd like
Functionality to archive a dataset to cold storage.
Additional context
None.