Question about CBT/Dirty Blocks

[madmax01]

@abbbi Hello,

the Program looks very interesting.

what i think is from quick look > its may complex to handle "Traditional Backups" when lots activities

• Full Backup
• then Diff or Inc

• then archive/retention

is there may any chance a Feature Request to an real "CBT" Modern Backup Method + Retention Logic.

CBT Modern Backup = its always a Full Backup... just new blocks getting backed up and merged into Base.

when restoring seeing only 1x backup set. .... rather into diff/inc complexity.

when restoring a VM from vmare for example i always see only 1 Image.

may kinda Idea can be done on Linux with this App also?.

[abbbi]

hi,

sorry, i dont think i completely understand your point here. In fact, virtnbdbackup does create backups based on CBT, and it does exactly what you are describing.

The first full backup will backup all required dirty blocks to a given directory, subsequent calls with backup mode incremental or differencial (to the same backup target directory) will only backup the changes since the last full or incremental (if it is supported by the libvirt/qemu and qcow image version you are using).

If you restore,it will automatically roll back the changes from all available backups (full + inc) and will create an qcow image with the latest contents.

The tool does however not cope with "retention" logic. The Retention is basically changing the backup target directory, if you want to create a new backup set (f+subsequent inc or diff backups), and i dont intend to implement retention handling at any point. Hiding the different backup modes or files just to simplify things from the users does not make any sense for me, after all backup is not a "setup once, works anyways" type of thing.

Of course you can build your own solution based on this utility which does all those kind of things, but i prefer the old way unix tools have been designed: do one thing, and do it properly.

Implementing a retention logic into this utility is lots of work and if things "go nuts" and some bug in the utility make you lose backups because the retention handling was buggy: i leave this point to the users. This utility is about saving and restoring blocks not about retention times (After all: retention handling is always tied to a backup concept and having a logic that meets the requirements of all users is somekind of a hard thing for me to imagine.).

[madmax01]

Hi,

if i understand the Process correctly logic would be

1x Full Backup and then from there Inc or Diff.

I would understand this as Traditional Backup. Would think when i have retention 2 weeks........ 1x full and rest inc or diff.... then i would understand the "full" never allowed to be corrupted because you cannot restore anymore having only deltas - or? If so then this would require to have multi "Full" Backups to avoid Risks. 1x per week or so.

also if its traditional. then restoring "inc" is slower as its just "delta" from last inc. diff is faster restore.

---

or may i missunderstand something here, apologize if so?

what i ment is - Custom Method would be then: You drive just a Backup.. don't need to flag if full or inc or diff....

first 1 is always Full because of init... and from there its automatically picking up just "deltas" from last backup. Difference now is > its merged into base.

if you want to restore > you would just say want to restore last backup from last Week Day X... and don't need to query if its full /diff or inc.

Benefit here is > Users don't need to take care about the "backup" method and to see which Risks behind. As you always have 1 Backup

regarding Retention. Would this mean its not possible to delete specific Dates out?

Not sure if there are too much differently Methods. for "Default Methods" Retention sure is interesting (Day/week/month/year etc,..)

Most known i think are

• CDP (Continuous data protection) > don't think its interesting for everyone -Dayli/weekly/monthly/Custom > i think such are most common
• GFS (grandfather/father/Son) > not sure if its common

based on Method the Retention would then kick into.. if its dayli and 2 weeky retention then based on "Backup Date" would may then simple rotate and delete out to maintain.

If retention is not available atm > hows possible to clean out manually? ;)

[abbbi]

1) you create a full backup

virtnbdbackup -d vm -l full -o /tmp/BACKUP

-> from this point on you can create incremental backups as long as you like to the same folder:

virtnbdbackup -d vm -l inc -o /tmp/BACKUP

-> only deltas are now saved. Of course you need to specify the backup mode, virtnbdbackup does currently not decide by itself to switch to mode incremental if there is already a full backup in /tmp/BACKUP/. This could be solved by introducing a backup mode called "auto" which switches to incremental if an full backup is already existant in the target directory.

See: https://github.com/abbbi/virtnbdbackup/issues/52

-> you can repeat that step for months, but what happens if your full backup gets corrupted after 3 weeks because you have an disk failure on your target backup? You lose the possibility to restore the complete virtual machine.

Thus it is of course recommended to take another full backup in certain time frames if you want to be sure your data is save. "Retention" thus is as simple as rotating the target directory and specifiying another backup mode.

[madmax01]

thx @abbbi

that's the Reason why i ment > it's may interesting to change the Behavior from Traditional Style away to modern.

in modern Way peoples don't need to care about if its full/diff or finc.... its always a Full Backup > because CBT does the Work and set shows always only 1x Full backups as all is merged. you may don't like to care about if you're "Restore" is save yes/no because you always have Backups X ;) .

is just an Idea.. Respect you're Way

[abbbi]

See: https://github.com/abbbi/virtnbdbackup#rotating-backups