search

plougher / squashfs-tools

tools to create and extract Squashfs filesystems
GNU General Public License v2.0
740 stars 191 forks source link

mksquashfs pipe #88

Open i0x71 opened 4 years ago

i0x71 commented 4 years ago

Hello, Is it possible to pipe put the output of mksquashfs to stdout ? I have half a terabyte of files that i would like to make into a squash image and pipe them into an s3 client, storing it in a local image file would take up unnecessary space.

Thanks

AnrDaemon commented 3 years ago

Try writing to /proc/self/fd/1

Dr-Emann commented 3 years ago

No, the first several bytes (the superblock) depend on the results of building the whole archive.

plougher commented 3 years ago

No, the output has to be seekable (which a pipe isn't).

This is first because the Squashfs superblock (which contains information about where things are placed in the filesystem and what is in it) is at the start of the filesystem and the contents are only known once the filesystem has been generated and written out. To update the superblock after the filesystem has been written out requires the output file to be seekable.

Second Mksquashfs does de-duplicating on the contents of the filesystem, and this again requires the output to be seekable, so Mksquashfs can read back previously written out files, and do a byte comparison.

There is way of doing it, but, it will require a reasonable amount of coding, and it got a lower priority to do than a lot of other requests. There is already a lot of new stuff in the next release, and it has taken a long time to test and document it.

So I intend to work on this feature after the next release (which be in the next couple of days).