search

wp-media / backwpup

BackWPup - WordPress Backup Plugin
https://backwpup.com
GNU General Public License v2.0
88 stars 35 forks source link

Split backup file into chunks #47

Open noplanman opened 7 years ago

noplanman commented 7 years ago

I'm running into an issue where the hosting has file size limitations, so a complete backup cannot be made in one go.

It would be really great if the compressed backup could be chunked into multiple files. The file size of each chunk could be defined in the settings.

widoz commented 5 years ago

Hi @noplanman can you elaborate a bit for example with which destination? The idea could be interesting but would mean change a lot of things internally.

noplanman commented 5 years ago

I was running into this with a WP installation that just had a LOT of uploaded content, thus making the final zipped backup file really big. Problem was that the hosting itself had a filesize limit, so the backup would fail when that size was reached.

Had to split the backup up into multiple jobs, which is not the prettiest solution as you can imagine.

This will be the case for all destinations, right? The temporary backup file first gets created on the drive, then gets send to the destinations before being deleted / moved.

widoz commented 5 years ago

I like the idea, there are some things we have to discuss in order to have a better idea of how could work. For example, the size of the chunk will be defined in the job settings, but what happen if a job is deleted? The informations of the chunk sizes cannot be retrieved, and we cannot use manifest.json to store those informations because the file have to be decompressed to extract the manifest.

Another thing is, what if you have to recover an installation but you don't have access to WordPress admin? How you can recreate the original backup?

noplanman commented 5 years ago

the size of the chunk will be defined in the job settings, but what happen if a job is deleted?

The split zip files would be backup.zip.001 with continuous number sequence for all files, so the size shouldn't matter, as it would just get all files and extract one by one.

UpdraftPlus does chunking too, so maybe check out how they solve this?

I'm just not sure if the chunks can be written on the fly or if the complete backup file needs to exist first, in which case it would hit the max filesize limit too...