mega-backup and incremental backups?

[guardiandigital]

Is it possible to use "backup" to create more of a backup schedule rather than just a time to do a full backup every time?

I'd like to be able to perform a full backup on Sunday, then incremental backups during other days. This would not only save on space required, but on the time it takes to perform a backup.

[guardiandigital]

@mattw-mega do you have any suggestions here?

[mattw-mega]

The concept with MEGAcmd's backup is to make a copy of the entire folder tree at that time, it doesn't have any idea about incremental steps. Files that are already in the cloud are cloned rather than being uploaded, in case that is the part you're worried about. It sound like your case would be met better by the 'sync' command, but that runs continuously, and also if you change something in the cloud then that same change will be applied locally.

[dwreski]

@mattw-mega the part I'm worried about is that each backup appears to consume the same size as that which is being backed up - so a 10GB directory requires 10GB on the remote system, despite very few changes from one day to the next.

Doing a full backup each day just isn't economical or efficient to transfer 10GB each day.

Can this be avoided?

[charlie39]

sync is what you are looking for

[dwreski]

No, I understand the differences between sync and backup. I want to create a series of incremental backups in case of local disk failure, ransomware, accidental deletion of files, etc., but want to do it as efficiently as possible.

Thanks so much for your help.

[charlie39]

then i guess you have create those backups in the local system itself with something like rsync and then upload those back ups

[dwreski]

@mattw-mega It would be awesome if "backup" could accept a filelist. Duplicating the content to be backed up locally before backing it up remotely, then having to figure out how to restore that, would be problematic.

I really don't see this as a real backup solution, sadly, because I love the interface.

[mattw-mega]

The backups will take storage space, but if not much has changed then it shouldn't take much upload bandwidth because the files that already have copies in the cloud will just be cloned, not uploaded.

[guardiandigital]

@mattw-mega can you help me understand what then I'm doing wrong? Each backup is consuming the same as the previous, resulting in the use of my full available space.

dave@mydomainl.com:/Documents$ du Documents_bk_202102*
FILENAME                                                                                                           SIZE
Documents_bk_20210207100611:                                                                                16498170519
Documents_bk_20210208060000:                                                                                16498170519
Documents_bk_20210209060000:                                                                                16498260541
Documents_bk_20210210060000:                                                                                 6990364229
-----------------------------------------------------------------------------------------------------------------------
Total storage used:                                                                                         56484965808

I am using the following command to create the backups:

mega-backup Documents/ Documents/ --period="0 0 11 * * *" --num-backups=10

It only made it through three backups because my 50GB available space was fully utilized.

[mattw-mega]

Yes each backup is a copy of all your files from that time, and they all take space. It's the upload bandwidth and time that is saved by cloning files that you already have somewhere in the cloud, but they still take up storage space.

[guardiandigital]

Are there any plans to develop more of a backup solution, not just a place to store files?

"backup" is kind of incorrect here - it's more of a "copy" or "upload", I would think.

[mattw-mega]

Yes there are but I can't share details, sorry

[guardiandigital]

Okay, an indication that it's being considered is enough for me for now :-)

I'd be happy to beta test....

[hilgenberg]

Hello @guardiandigital Did this ever go anywhere? I have the exact same problem.

[guardiandigital]

@hilgenberg no, only what's been reported above, sadly. Maybe I'll check again in a few months.

[Alex-Bujorianu]

I too would like to see incremental backups by default in megacmd. I am not very happy with the current implementation.

[rsbrux]

Is anything happening here? @guardiandigital volunteered to beta test, but mega-backup is already marked beta, so I guess we are all beta testers ;-)
For sync to be an alternative, it would need to have an option for one-way synchronization (upload only). Furthermore, incremental backups shouldn't be that difficult to implement. Couldn't they be based on rsync or rsnapshot?

[doughnet]

how is this not implemented yet? It seems so simple and should be standard.

In rclone you can use the "delete" command and provide an argument like --min-age 60d which will delete files in a folder that are older than 60 days...

[kpaulisse]

As someone evaluating this service, the behavior of "backup" was quite surprising, leading me to find this issue. Since it's now been a year since the last post in either this issue or https://github.com/meganz/MEGAcmd/issues/638, is there any indication whether this is still under consideration and/or when it might be added?

[sh1boot]

The only problem is that duplicates take space, even though they're identified as duplicates. That means a large file that doesn't change but exists in the backed-up tree eats all your quota and then your backups don't work anymore.

So why do duplicates have to behave that way?