lsst-uk / csd3-echo-somerville

Code to backup from CSD3 to Echo S3, curate at STFC cloud and expose to Somerville
Apache License 2.0
0 stars 0 forks source link

Add options to read and write lists of files and folders to backup. #72

Closed davedavemckay closed 1 month ago

davedavemckay commented 1 month ago

Scanning the filesystem and creating lists of files and folders to (zip and) upload takes a while and is necessarily serial.

On continuation runs it would be useful to read these from files.

Use json to serialise/deserialise dict objects.

davedavemckay commented 1 month ago

Currently in CLI options list, but tagged "NOT YET IMPLEMENTED"