plougher / squashfs-tools

tools to create and extract Squashfs filesystems
GNU General Public License v2.0
752 stars 193 forks source link

Extracting and then re-creating squashfs archives #98

Closed keeely closed 3 years ago

keeely commented 3 years ago

The pseudo file options allow one to create block devices, files, directories and (with v4.4) symlinks. However they don't allow one to specify the file timestamp as far as I can see, so you have to have (presumably) whatever time the input file or directory has. If the file is dynamically generated, I'm going to assume it takes the system time at the point it was generated. This may not be what you want, so the ability to specify timestamp in the pseudo-file specification line would be a welcome feature.

I could find no easy way of generating a file full of pseudo-file entries from an existing squashfs, so I started writing something to do so. Then it struck me someone must have already written this before. Has someone done this?

plougher commented 3 years ago

You raise some interesting issues. First whether it should be possible to (re-)create a Squashfs filesystem from pseudo file definitions. Second, the lack of timestamp information in pseudo definitions makes it impossible to recreate a filesystem using pseudo definitions as timestamping cannot be preserved.

As I felt this is a useful thing to do, I spent some time adding the necessary functionality. This additional functionality is split into three areas:

  1. Adding timestamps. The pseudo definitions have been extended to add timestamps. New upper-case pseudo definitions have been added which take timestamps. For example, the new set of pseudo definitions is

                            filename d mode uid gid
                            filename m mode uid gid
                            filename b mode uid gid major minor
                            filename c mode uid gid major minor
                            filename f mode uid gid command
                            filename s mode uid gid symlink
                            filename i mode uid gid [s|f]
                            filename l filename
                            filename L pseudo_filename
                            filename D time mode uid gid
                            filename M time mode uid gid
                            filename B time mode uid gid major minor
                            filename C time mode uid gid major minor
                            filename F time mode uid gid command
                            filename S time mode uid gid symlink
                            filename I time mode uid gid [s|f]
                            filename R time mode uid gid length offset

where the upper-case versions take a timestamp. This can be specified as an absolute numeric value, which is taken to be the number of seconds since the epoch of 00:00:00 UTC on 1 January 1970. A string can also be used, which is passed to the "date" command for interpretation, and can be anything that "date" considers a valid date.

  1. Adding pseudo definitions. The previous pseudo definitions lacked the ability to create hard-links, and also lacked the ability to recreate regular files, as there was no capability to store data in a pseudo definition.

Without that ability it is impossible to recreate a Squashfs filesystem using pseudo definitions. These pseudo definitions have been added.

  1. An option has been added to Unsquashfs which will produce a "pseudo file" definition of the input Squashfs filesystem. This pseudo-file can be passed to Mksquashfs to re-create the Squashfs filesystem.

Usage:

% unsquashfs -pseudo-file name squashfs-filesystem

This will create a pseudo file called "name", which is a representation of the supplied "squashfs-filesystem".

% mksquashfs - squashfs-filesystem -pf name

This will take the pseudo-file "name", and it will produce the squashfs filesystem called "squashfs-filesystem". This will be identical in content to the original filesystem supplied to Unsquashfs.

keeely commented 3 years ago

This is great news and will allow me to simplify some scripting that achieves the same by using the sqfs2tar utility from squashfs-tools-ng (the only way I could previously do this).

One other thing, I don't know if you realised there is a standard - of sorts - for this file spec information in the Linux kernel: https://www.kernel.org/doc/Documentation/filesystems/ramfs-rootfs-initramfs.txt, example:

  dir /dev 755 0 0
  nod /dev/console 644 0 0 c 5 1
  nod /dev/loop0 644 0 0 b 7 0
  dir /bin 755 1000 1000
  slink /bin/sh busybox 777 0 0
  file /bin/busybox initramfs/busybox 755 0 0
  dir /proc 755 0 0
  dir /sys 755 0 0
  dir /mnt 755 0 0
  file /init initramfs/init.sh 755 0 0

It doesn't support your command output capability for 'f/F'. Maybe you had a reason to come up with a different format and that's fine (a script to convert between is trivial) just thought you should know.

Thank you for your support for this software.

plougher commented 3 years ago

One other thing, I don't know if you realised there is a standard - of sorts - for this file spec information in the Linux kernel: https://www.kernel.org/doc/Documentation/filesystems/ramfs-rootfs-initramfs.txt, example:

  dir /dev 755 0 0
  nod /dev/console 644 0 0 c 5 1
  nod /dev/loop0 644 0 0 b 7 0
  dir /bin 755 1000 1000
  slink /bin/sh busybox 777 0 0
  file /bin/busybox initramfs/busybox 755 0 0
  dir /proc 755 0 0
  dir /sys 755 0 0
  dir /mnt 755 0 0
  file /init initramfs/init.sh 755 0 0

It doesn't support your command output capability for 'f/F'. Maybe you had a reason to come up with a different format and that's fine (a script to convert between is trivial) just thought you should know.

I first added pseudo file support in April 2009. At that point I was unaware of "gen_init_cpio" and the above file format.

The initial pseudo file support was brought about to help people create block/char devices where they didn't have root access to create them in the source filesystem. As such it was something I came up with and implemented from scratch without looking at any previous ways of doing it. So choosing a different format wasn't deliberate.

Thank you for your support for this software.