tasket / wyng-backup

Fast backups for logical volumes & disk images
GNU General Public License v3.0
251 stars 16 forks source link

Metadata: Scan only what's needed #201

Open tasket opened 5 months ago

tasket commented 5 months ago

Problem

Wyng currently scans all archive metadata items (root, volumes and sessions) on startup. This impacts startup times and is also sub-optimal when encountering or recovering from errors.

Solution

Map the commands to metadata requirements, such as receive only needing one volume's metadata at a time. When receiving older sessions, it may not be necessary to instantiate all session objects / infos. verify, diff and list could have the same/similar metadata usage profile, while delete would be a special case where as little as zero volinfos could be used if the user is able to specify a volume id. And monitor would be another special case, since it is a goal to be able to run it without authenticating.

The profile for send would not be as flexible as the others, since the send workflow may call on both deduplication (which requires every bit of metadata) and pruning operations.

The arch-check command could have a similar profile to verify depending on the parameters used.

Notes

A generic load() method for objects may provide needed flexibility.

Its probable this issue will focus mainly on session objects, although being flexible with volume object should also be a goal.