search

RichardMidnight / pi-safe

Create your own custom image files
GNU General Public License v3.0
224 stars 17 forks source link

tar compression? #18

Closed Botspot closed 3 years ago

Botspot commented 3 years ago

Could you add the ability to compress the sd card as a .tar archive? Once that's working, it ought to be fairly simple to also add tar.xz and tar.gz support.

RichardMidnight commented 3 years ago

Thanks for sharing the idea. I do think it could be done, but I am not sure of the value of doing it.

As I understand it, tar (tape archive) is good at archiving files but would struggle with the boot sector and partition table. I think we would need a tool to read the boot sector and partition table into a file, then to write them back out to the device for restore.

The dd command PiSafe is based on handles the boot sector and partition table automatically. I agree that compressing a tar file with xz or gz is fairly simple.

TotalCoolness commented 3 years ago

raspibackup does tar.

Botspot commented 3 years ago

raspibackup does tar.

@TotalCoolness, thanks for the lead. Unfortunately, the way raspibackup is using tar would not work for my purposes. I need a way for tar to compress a single byte stream, be it from a block device or from a URL. Raspibackup uses tar, but it is compressing many individual files which is useless for what I need it to do.

RichardMidnight commented 3 years ago

Hotspot: Tell me what you are trying to do. Would taring the .img file help? The result might be backup.img.tar.zip

Botspot commented 3 years ago

Hotspot: Tell me what you are trying to do. Would taring the .img file help? The result might be backup.img.tar.zip

I'll start from zero. Basically, I'm designing a tool of my own and it needs to compress disk images, disks, and stdin data streams to the tar format. The last requirement means that I'm forced to use a syntax like this:

cat /dev/sda | tar

That does not work, but I'm convinced it's possible.

RichardMidnight commented 3 years ago

OK, try this and tell me if it is close to what you want. I know it is not using stdin, but your answer will help me understand the structure of the file you want created.

1) use pisafe to create a backup named backup.img 2) from a terminal in the Downloads folder type in "tar -cf backup.img.tar backup.img". this creates backup.img.tar

Botspot commented 3 years ago

OK, try this and tell me if it is close to what you want. I know it is not using stdin, but your answer will help me understand the structure of the file you want created.

  1. use pisafe to create a backup named backup.img
  2. from a terminal in the Downloads folder type in "tar -cf backup.img.tar backup.img". this creates backup.img.tar

The example you gave does what I'm looking for it to do, in the sense that the tar archive "contains" a single .img file inside its compressed filesystem. As I mentioned earlier though, it needs to be from stdin. And no, using a tempfile is not allowed:

cat /dev/sda > /my/big/file.img
tar -cf /my/big/file.img.tar /my/big/file.img
RichardMidnight commented 3 years ago

Hmmm... I am not aware of tar accepting a DataStream from stdin; it wants filenames. You can specify the filenames via stdin, with "-T -"but I do not know how to specify the DataStream from stdin. Someone seems to have made it work with a python script. https://askubuntu.com/questions/587013/how-can-i-make-tar-read-input-from-stdin . Maybe that will help.