Open cniethammer opened 3 years ago
yes, also thinking about that.
there is many questions, would this be a tree? or an archive? should there be some book keeping? e.g. should the workspace DB entry still exist with a pointer to the data and be recoverable? (and ofc not expire)
I think the archiving format is likely storage target specific. For HPSS I assume that one wants to use htar or another natively supported method?
I do not see that the workspace tools should do bookkeeping of the archived data in its DB after it is successfully archived, even though it looks tempting. There would be more questions and complications in this case: E.g., a ws storage system may get upgraded - and in this case you do not want to move all this old DB stuff with you. Or, one allows to restore a workspace on another system from a common archive storage, so should this be synced then?
However one could think about a query method that allows the ws tools to get information from the archiving storage about archived workspaces - but this might already go into the direction of data managment services like EUDAT, etc.
I'd like to see a
ws_archive
command that provides an easy and efficient way for archiving of a workspace to a long time storage system, e.g. HPSS.Data movement could be implemented as an archiving deamon, which performs the data movement based on workspace file system and target storage system load.
Ideas for command usage/options