Support for FAT systems?

[Bluscream]

Couldn't you just split the archive each 4GB's ?

[framps]

From a technical point of view it shouldn't be a big deal to get this implemented for dd and tar backup type :wink: But there are some issues which have to be taken into account:

1) backup time will increase 2) enough ext3/4 temp space has to be available to split the backup file and to join the backup pieces during restore. This means there has to be an ext3/4 partition available to receive the backup file which then will be split into pieces and copied to the fat backup partition. This could be either the SD card if it's large enough or an additional attached SSD or flash drive.

From a coding and test point of view this feature will need a reasonable effort. That's why I ask for the number of raspiBackup users which actually need and will use this feature. Why do you think a lot of raspiBackup users need and will use this feature?

Please keep in mind you can add this feature yourself. Just write a wrapper script which calls raspiBackup and creates the backup on some temp EXT3/4 partition and finally splits the backup file and copies it to the fat backup partition. During restore the reverse procedure has to be executed - join the backup on some EXT3/4 backup partition and let raspiBackup restore the backup.

[Bluscream]

Thanks for your response.

backup time will increase

I dont think so. Compression is the only thing that could incrementially increases the duration of backing up. Compressing a file on windows using 7zip takes roughly the same amount of time with split archives or without. The only thing that can really increase the duration is the network bandwith when saving to a mounted NAS. That's however not really part of this request and should be okay given the fact that home networks become quite fast lately even wireless with the device manufactoring racing to implement the latest and greatest 1337.acntw technologies.

enough ext3/4 temp space has to be available to split the backup file

Sounds weird. 7zip can split on the fly. it doesn't need to have the whole archive + the splitted chunks at the same time. Maybe i'm misunderstanding something there :thinking:

to join the backup pieces during restore

When something goes so horribly wrong that you have to restore a full backup i would always go the safe route of restoring the image on another machine via an SD card reader (infact thats how i make my backups [still because no FAT support]) creating backups on the other hand is something you should do frequently (which i and maybe others don't do so often because shutting down machine, ejecting sdcard, using sd card reader, leaving the machine useless while the backup occurs, etc...) thats something i'd prefer to be done without interaction

Please keep in mind you can add this feature yourself.

That would also mean i'd have to maintain it and i have like 50000000 other things on my Pi to maintain. I would rather know that's been done by someone who actually knows what he's doing.

[framps]

Sounds weird. 7zip can split on the fly.

Right now raspiBackup uses gzip to compress the dd and tar backups. I frankly haven't had 7z on my radar until now. I also didn't know 7z supports split and join and dd or tar can just pipe their result into 7z which handles all the split/join stuff. So your feature request actually is also a request to support 7z :wink:

When something goes so horribly wrong that you have to restore a full backup

raspiBackup is primarily designed to restore the whole backup all the time. But it's indeed also possible to restore parts of the backup. But this has to be done manually without any support by raspiBackup. If you use rsync backup it's quite easy because the backup is just a directory tree. But this requires ext3/4 ...

What's the reason you use a FAT partition for your backups? Windows boxes use NTFS which don't have the 4GB limit.

[Bluscream]

Right now raspiBackup uses gzip to compress the dd and tar backups.

I would be surprised if the other archiving softwares wouldn't act the same. On windows for example WinRAR can do everything 7-zip can, even open it's 7z format

What's the reason you use a FAT partition for your backups

Because i use a SMB NAS. I know modern NAS systems support all kinds of exotic files systems, but rule of thumb is FAT32 has the highest compability there. And i think more than 60% of home NAS systems use it.

[framps]

I would be surprised if the other archiving software wouldn't act the same

On Linux you can pipe tar or dd output into split and vice versa. So you are right :smiley:

rule of thumb is FAT32 has the highest compatibility there. And i think more than 60% of home NAS systems use it.

Every NAS I know about supports nfs which allows to use rsync backup and is the most efficient backup method. Windows user use NTFS on their NAS instead of FAT. I doubt there are so many people using FAT on their NAS.

Anyhow - you convinced me it's much easier to split/join the backups than I thought initially. I will try to create a prototype to get an idea how much effort is needed for this feature. I will not use 7z but gzip together with split. Just in case I'm able to create a prototype I want you to verify it's functionality. Which backup type is the one you are looking for primarily: dd or tar?

[Bluscream]

Which backup type is the one you are looking for primarily

Hmm tbh idk, in the end i'm gonna restore it with etcher, so i'd prefer a type that it supports.

[framps]

Hmm, as I wrote before - raspiBackup handles full backup and restore. If the dd backup (that's what you need for etcher) is split you have to join the pieces yourself on windows before you can use etcher. All this will be handled by raspiBackup during restore.

I know a lot of people use a dd backup and either win32diskimager or etcher to restore the backup for convenience but frankly I don't understand why they don't use raspiBackup instead.

[Bluscream]

Everytime i had to restore an image onto a Pi the device was broken enough to not boot. RpiBackup wont help there ^^

[framps]

That's why I suggest to have a spare SD card primed with Raspbian and use this image to restore a backup. Anyhow, I see it's convenient for a Windows user 😃 Will keep you posted about the prototype.

[framps]

@Bluscream I just created a new branch called split. There you find new code which accepts option --split and splits the generated backup file into chunks of the size of the --split parameter. Please note the split function is only implemented for type dd with no compression (option -z) for now.

To create a backup on a fat filesystem use sudo raspiBackup.sh --split 4G -t dd /backup

To restore the backup use sudo raspiBackup.sh -d /dev/sd<x> <backupDirectory>

If you detect any issues just provide the backup and/or restore debug log.

[framps]

@Bluscream You requested a feature for raspiBackup and I created a prototype three weeks ago. No feedback from your side until now :disappointed:

[Bluscream]

Sorry for that. I just haven't done any "risky" operations which would have required a backup till now. But that's going to change because i'll add a supereval command to my telegram bot which allows anyone to eval scrambled code on my server :D

[framps]

Closing this issue because of no requestor support

[Bluscream]

rip :(

[Bluscream]

i will test it tomorrow if you remind me, i promise :)

how do i update rapsibackup to the changes you made?

[framps]

@Bluscream I just deleted the branch I created to support for your feature request. As you can see I'm willing to add features to raspiBackup but I also expect some activity from your side.

[Bluscream]

Well, you didnt tell me how :/

[framps]

I'm sorry but you're the first one who didn't manage to find out how to extract a file from a git branch. There are a bunch of ways to get this done. I miss some own initiative on your side.