Ability to view files in the archive

[adamantida]

Hello, I have questions about viewing files in archives using the dwarfs program. In zpaqfranz there is a command "l" for this, but in dwarfs there is no such command (at least it is not mentioned anywhere). However, when using dwarfsck -d 4 it appears to be able to view files, but the results look strange. Are there plans to add a special command to view archive files in dwarfs in the future? Also wondering if there will be support for archive versions similar to zpaqfranz in future dwarfs updates?

[mhx]

Hi and thanks for your feedback!

I guess that because DwarFS is primarily a file system rather than an archive format (although it seems that it's quite often used as such), this request has never really come up before. It'd be trivial to implement and I'll probably just do it if it's considered useful.

However, there is a way to already do this, but it's a bit of a hack:

$ dwarfsextract -i min.dwarfs -f mtree
I 08:00:56.986420 extraction finished without errors
#mtree
./bench.sh time=1616939327.0 mode=644 gid=100 uid=1000 type=file size=33
./foobar time=1606400164.0 mode=777 gid=100 uid=1000 type=link link=foo/bar
./format.sh time=1606256045.0 mode=755 gid=100 uid=1000 type=file size=94
./perl-exec.sh time=1606161908.0 mode=644 gid=100 uid=1000 type=file size=87
./dev time=1606845311.0 mode=755 gid=100 uid=1000 type=dir
./empty time=1606945040.0 mode=755 gid=100 uid=1000 type=dir
./empty/alsoempty time=1606945040.0 mode=755 gid=100 uid=1000 type=dir
./foo time=1616013726.0 mode=755 gid=100 uid=1000 type=dir
./foo/bad time=1606814322.0 mode=777 gid=100 uid=1000 type=link link=../foo
./foo/bar time=1606310580.0 mode=644 gid=100 uid=1000 type=file size=0
./foo/bla.sh time=1616939377.0 mode=644 gid=100 uid=1000 type=file size=33
./foo/1 time=1616013804.0 mode=755 gid=100 uid=1000 type=dir
./foo/1/fmt.sh time=1606256045.0 mode=755 gid=100 uid=1000 type=file size=94
./foo/1/2 time=1616013816.0 mode=755 gid=100 uid=1000 type=dir
./foo/1/2/xxx.sh time=1606256045.0 mode=755 gid=100 uid=1000 type=file size=94
./foo/1/2/3 time=1616939417.0 mode=755 gid=100 uid=1000 type=dir
./foo/1/2/3/copy.sh time=1616013831.0 mode=755 gid=100 uid=1000 type=file size=94

This uses libarchive's mtree format as output, which will simply list all files. While this works, the overhead is massive, as it'll decompress all the data even though none of it is used. The fastest way to list all files currently is to just mount the image.

The output of dwarfsck is really meant for debugging and seeing what's going on in the file system metadata. It's not meant for casual consumption. :)

Also wondering if there will be support for archive versions similar to zpaqfranz in future dwarfs updates?

Yes, but there's no firm timeline. This has been a request for a long time already, see also #18.

[mhx]

@adamantida, the next release will have a -l / --list option for dwarfsck:

$ dwarfsck ../test/data.dwarfs -l 
foobar
format.sh
perl-exec.sh
test.py
unicode
unicode/我爱你
unicode/我爱你/☀️ Sun
unicode/我爱你/☀️ Sun/Γειά σας
unicode/我爱你/☀️ Sun/Γειά σας/مرحبًا
unicode/我爱你/☀️ Sun/Γειά σας/مرحبًا/⚽️
unicode/我爱你/☀️ Sun/Γειά σας/مرحبًا/⚽️/Карибського

$ dwarfsck ../test/data.dwarfs -lv
---lrwxrwxrwx 1000/1000    7 2020-11-26 15:16 foobar -> foo/bar
----rwxr-xr-x 1000/1000   94 2020-11-24 23:14 format.sh
----rw-r--r-- 1000/1000   87 2020-11-23 21:05 perl-exec.sh
----rw-r--r-- 1000/1000 1012 2020-12-05 22:46 test.py
---drwxr-xr-x 1000/1000    1 2023-06-28 10:26 unicode
---drwxr-xr-x 1000/1000    1 2023-06-28 10:26 unicode/我爱你
---drwxr-xr-x 1000/1000    1 2023-06-28 10:26 unicode/我爱你/☀️ Sun
---drwxr-xr-x 1000/1000    1 2023-06-28 10:26 unicode/我爱你/☀️ Sun/Γειά σας
---drwxr-xr-x 1000/1000    1 2023-06-28 10:26 unicode/我爱你/☀️ Sun/Γειά σας/مرحبًا
---drwxr-xr-x 1000/1000    1 2023-06-28 10:29 unicode/我爱你/☀️ Sun/Γειά σας/مرحبًا/⚽️
----rw-r--r-- 1000/1000    8 2023-06-28 10:29 unicode/我爱你/☀️ Sun/Γειά σας/مرحبًا/⚽️/Карибського

[adamantida]

Perhaps it will be useful. I want to say about the inconvenience of using --tool=dwarfsck in "universal" is too long and inconvenient, instead it is more convenient, in my opinion, to use short designations in the form of letters without "-", let's say there is dwarfsextract, the name speaks for itself, it is convenient to use as "./dwarfs x -i min.dwarfs -f mtree", as well as "./dwarfs x min.dwarfs -f mtree". This is more convenient and easier to use. I also noticed that when using universal dwarfs on windows --tool=dwarfs - an error pops up, which is natural without the .dll But in normal bin for using ./dwarfs it doesn't pop up.

[mhx]

Perhaps it will be useful. I want to say about the inconvenience of using --tool=dwarfsck in "universal" is too long and inconvenient, instead it is more convenient, in my opinion, to use short designations in the form of letters without "-", let's say there is dwarfsextract, the name speaks for itself, it is convenient to use as "./dwarfs x -i min.dwarfs -f mtree", as well as "./dwarfs x min.dwarfs -f mtree". This is more convenient and easier to use.

I'm not a big fan of these single letter commands. I agree that dwarfs-universal --tool=dwarfsck is verbose, but then again, you have the choice of using symlinks, you can just create copies of the universal binary with different names, or you can install the regular binaries.

I also noticed that when using universal dwarfs on windows --tool=dwarfs - an error pops up, which is natural without the .dll But in normal bin for using ./dwarfs it doesn't pop up.

Can you elaborate?

Without the WinFsp DLL in the PATH, running

> dwarfs-universal --tool=dwarfs

will show a message in the terminal:

failed to load winfsp-x64.dll

Whereas running

> dwarfs

will show a pop-up dialog saying that winfsp-x64.dll could not be loaded.

Once I put the WinFsp DLL in the PATH, both tools work just fine.

[adamantida]

Can you elaborate?

Yeah, sure

The dialog box itself is missing. The error is not displayed that winfsp-x64.dll could not be loaded

Once I put the WinFsp DLL in the PATH, both tools work just fine.

I understand that perfectly well, but I'm talking about the fact that the error window itself doesn't appear exactly when using ".\dwarfs.exe"

[mhx]

Looks like it's a PowerShell specific issue...

[mhx]

Indeed: https://github.com/PowerShell/PowerShell/issues/16468

[mhx]

I've added a workaround for the power shell issue by using delayed loading also for dwarfs.exe; that will avoid the pop-up and rather show the same error message as when using the universal binary. This will be part of the next release.