search

Corsinvest / cv4pve-barc

Backup And Restore Ceph for Proxmox VE
https://www.cv4pve-tools.com
GNU General Public License v3.0
82 stars 22 forks source link

Use borg backup #25

Closed mgiammarco closed 6 years ago

mgiammarco commented 6 years ago

I know this is a big change but I ask it anyways. The standard proxmox backup is full only so it is not very helpful. Your solution is great because it saves only diffs so I can maintain a long history. There are two remaining problems: 1) if I have, for example, 40 backups, and I start a restore it is very long because it need to start from first file and then merge all 40 files over it 2) if a vm saves a lot of data I have big diffs and so I can keep short history.

Using borg to save snapshots will solve point 1, point 2 and also deduplicate data saving even more space to keep even more history.

franklupo commented 6 years ago

Hi, I use ZFS with deduplication. Is not a good idea?

Best regards

franklupo commented 6 years ago

Hi,

if I have, for example, 40 backups, and I start a restore it is very long because it need to start from first file and then merge all 40 files over it

per migliorare importazione si setta --rbd-concurrent-management-ops ma il rischio poi e di rallentare tutto.

Best regards

franklupo commented 6 years ago

Hi,

if I have, for example, 40 backups, and I start a restore it is very long because it need to start from first file and then merge all 40 files over it

How can you speed up this with borg?

Best regards

mgiammarco commented 6 years ago

I will not use incremental backups. Borg saves space because it has a new style deduplication engine that is used also as a diff/incremental backups strategy. So I can ask borg to restore directly latest image and not build it replaying 40 diffs.

2018-02-12 19:17 GMT+01:00 Daniele Corsini notifications@github.com:

Hi,

if I have, for example, 40 backups, and I start a restore it is very long because it need to start from first file and then merge all 40 files over it

How can you speed up this with borg?

Best regards

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/EnterpriseVE/eve4pve-barc/issues/25#issuecomment-365012261, or mute the thread https://github.com/notifications/unsubscribe-auth/AAIUFqDeRHvAijlvemLCwC61vGbMhfsmks5tUIAkgaJpZM4R4Hv8 .

franklupo commented 6 years ago

You must always export the whole image. I must verify. Backy2 maybe has something better.

mgiammarco commented 6 years ago

Yes unfortunately you must always export whole image. I will check backy2 too.

2018-02-13 11:29 GMT+01:00 Daniele Corsini notifications@github.com:

You must always export the whole image. I must verify. Backy2 maybe has something better.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/EnterpriseVE/eve4pve-barc/issues/25#issuecomment-365221145, or mute the thread https://github.com/notifications/unsubscribe-auth/AAIUFnn_dtaVQcPbDoIcwcpz8Nu2PW9Vks5tUWQJgaJpZM4R4Hv8 .

mgiammarco commented 6 years ago

I have looked at backy2 and I aeree it is better than borg because it is rbd specific

Il mar 13 feb 2018, 11:29 Daniele Corsini notifications@github.com ha scritto:

You must always export the whole image. I must verify. Backy2 maybe has something better.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/EnterpriseVE/eve4pve-barc/issues/25#issuecomment-365221145, or mute the thread https://github.com/notifications/unsubscribe-auth/AAIUFnn_dtaVQcPbDoIcwcpz8Nu2PW9Vks5tUWQJgaJpZM4R4Hv8 .

franklupo commented 6 years ago

I'm thinking of a version with integrated backy2. I'll think about it.

Best regards

mgiammarco commented 6 years ago

I have also looked at rdiff-backup and rsnapshot for their reverse diff protocol. Unfortunately they are not integrated with ceph like backy2 that is the Best choice

Il mar 13 feb 2018, 11:29 Daniele Corsini notifications@github.com ha scritto:

You must always export the whole image. I must verify. Backy2 maybe has something better.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/EnterpriseVE/eve4pve-barc/issues/25#issuecomment-365221145, or mute the thread https://github.com/notifications/unsubscribe-auth/AAIUFnn_dtaVQcPbDoIcwcpz8Nu2PW9Vks5tUWQJgaJpZM4R4Hv8 .