Storing arbitrary metadata in a backup for use in other tooling

[brownleej]

There are some operations on backups that require a connection to the cluster the backup is for. One example of this is expiring data by timestamp. If someone is building tooling that is going to operate on lots of backups for lots of different clusters, they need a way to determine which cluster is using which backup. Right now the only metadata we have to make this kind of determination is the backup name. This may not be enough information for someone whose system has a multi-level hierarchy that they use to manage clusters. As an alternative, could we provide a mechanism to provide arbitrary metadata when starting a backup and surfacing that metadata through the backup describe command?

[satherton]

One possible form this could take:

• Metadata refers to a set of named attributes, where the attribute name and attribute value are both strings
• IBackupContainer gets getAttributes() and setAttributes() methods.
• getAttributes() returns all attributes
• setAttributes() replaces the set of attributes with a new set of attributes
• IBackupContainer could also get readAttribute(), setAttribute(), and clearAttribute() to operate on individual attributes, which will have a default implementation that uses get/set attributes() to do a (non-atomic) read/modify/write.
• Attributes will be internally stored as JSON within the backup
• fdbbackup start will accept attributes to store in the backup upon creation
• fdbbackup describe will print the attributes found in a backup
• fdbbackup attributes will allow retrieval and modification of attributes for a backup URL, either as a whole or on a per-attribute basis for one or more attributes