search

fox-it / dissect.target

The Dissect module tying all other Dissect modules together. It provides a programming API and command line tools which allow easy access to various data sources inside disk images or file collections (a.k.a. targets).
GNU Affero General Public License v3.0
43 stars 44 forks source link

target-mount: only one partition will be mounted #236

Closed ghost closed 1 year ago

ghost commented 1 year ago

Given the following structure inside an EWF image:

part_00100000: Linux rev 1.0 ext4 filesystem data
part_04100000: Linux swap file
part_41bc5800: Linux rev 1.0 ext4 filesystem data
part_c1bc5800: Linux rev 1.0 ext4 filesystem data

If I mount the EWF image with target-mount only volume part_41bc5800 will be mounted in ./fs - part_c1bc5800 will be ignored. Is this intended behaviour?

Here's part of the output in verbose mode:

.....
Opened filesystem: <ExtFilesystem> on <Volume
 name='part_c1bc5800' size=116782729216 fs=<ExtFilesystem>> [dissect.target.target]
LVM volumes found: [] [dissect.target.target]
Encrypted volumes found: [] [dissect.target.target]                                                                                                                                                                                                                             
2023-04-24T18:42:28.936072Z [debug    ] Plugin registered: dissect.target.plugins.apps.av.trendmicro.TrendMicroPlugin [dissect.target.plugin] 
2023-04-24T18:42:28.941927Z [debug    ] Plugin registered: dissect.target.plugins.apps.av.mcafee.McAfeePlugin [dissect.target.plugin] 
2023-04-24T18:42:28.943873Z [debug    ] Plugin registered: dissect.target.plugins.apps.containers.docker.DockerPlugin [dissect.target.plugin] 
2023-04-24T18:42:28.945159Z [debug    ] Plugin registered: dissect.target.plugins.apps.webservers.webservers.WebserverPlugin [dissect.target.plugin] 
...

Volume c1bc5800 is not encrypted and can be mounted with sudo mount -o ro,norecovery mnt2/volumes/part_c1bc5800 ./mnt3/ without any error...

Schamper commented 1 year ago

It looks like target-mount currently only mounts the "root filesystem". We could definitely add mounts for all filesystems.

By the way, is this other partition used in /etc/fstab? If so, it's likely available at the configured mount point in there. Of course that doesn't take away that it's still a good idea to add mounts for all filesystems.

ghost commented 1 year ago

Yes the partition is used in /etc/fstab but doesn't show up in the corresponding folder.

(sorry misclicked and closed the issue xD)

Schamper commented 1 year ago

And there are files on it? That's weird. Are you able to share the contents of /etc/fstab? And perhaps some debug logs filtered on the parsing of fstab:

target-shell /path/to/file.E01 -vvvvv 2>&1 | fgrep -e Mounting -e Skipped -e Unsupported

In the meantime I've already created a PR to add mountpoints for all filesystems at #237!

ghost commented 1 year ago

Yes different files and folders, which can be inspected after a manual mount.

volumes:

part_00100000: Linux rev 1.0 ext4 filesystem data, UUID=ec76cc55-... (needs journal recovery) (extents) (large files) (huge files)
part_04100000: Linux swap file, 4k page size, little endian, version 1, size 252612 pages, 0 bad pages, no label, UUID=140edfd0-....
part_41bc5800: Linux rev 1.0 ext4 filesystem data, UUID=78b46bd6-... (needs journal recovery) (extents) (large files) (huge files)
part_c1bc5800: Linux rev 1.0 ext4 filesystem data, UUID=faedf1df-... (needs journal recovery) (extents) (large files) (huge files)

/etc/fstab:

UUID=ec76cc55-.... /boot    auto defaults   1 2
UUID=140edfd0-.... swap     swap defaults,pri=1 0 0
UUID=78b46bd6-.... /        auto defaults   1 1
UUID=faedf1df-.... /var     auto defaults   1 1

Running target-shell did not return anything, but looking a the logs I could find this lines for part_c1bc5800 only:

2023-04-25T13:28:56.764759Z [debug    ] <Target xxx.e01>: Opened filesystem: <ExtFilesystem> on <Volume name='part_c1bc5800' size=116782729216 fs=<ExtFilesystem>> [dissect.target.target]          
2023-04-25T13:28:56.764842Z [debug    ] <Target xxx.e01>: LVM volumes found: [] [dissect.target.target]                                                                                             
2023-04-25T13:28:56.764892Z [debug    ] <Target xxx.e01>: Encrypted volumes found: [] [dissect.target.target]

@ PR: Thanks :)

Schamper commented 1 year ago

I know the issue is closed by the PR, but I'll try and recreate your setup to see if I can replicate the issue. I might come back with some commands for you to try to aid in debugging :smile:

mischw commented 1 year ago

Mounting all filesystems should be implemented with dissect.target 3.9, correct? I tried it with some VMs and I couldn't get it to work the way I expected.

dissect.target is on version 3.9:

pip list | grep dissect.target
dissect.target     3.9

a) Debian VM / = ext4 /boot = ext2 /home = ext4

target-mount -vvvvv /mnt/c/WSL/debpart/5.vdi /mnt/TEMP 2>&1 | fgrep -e Mounting -e Skipped -e Unsupported
2023-06-20T13:47:27.899933Z [warning  ] <Target /mnt/c/WSL/debpart/5.vdi>: Unsupported mount device: /dev/sr0 /media/cdrom0 [dissect.target.target]

Only the root filesystem is mounted

b) Identical set up Debian VM, but different filesystems / = xfs /boot = xfs /home = xfs

target-mount -vvvvv /mnt/c/WSL/debpart/6.vdi /mnt/TEMP 2>&1 | fgrep -e Mounting -e Skipped -e Unsupported
2023-06-20T13:48:45.971356Z [debug    ] <Target /mnt/c/WSL/debpart/6.vdi>: Mounting <Volume name='part_00100000' size=999292928 fs=<XfsFilesystem>> at /boot [dissect.target.target]
2023-06-20T13:48:45.971516Z [debug    ] <Target /mnt/c/WSL/debpart/6.vdi>: Mounting <Volume name='part_28fb00000' size=10473177088 fs=<XfsFilesystem>> at /home [dissect.target.target]
2023-06-20T13:48:45.971658Z [warning  ] <Target /mnt/c/WSL/debpart/6.vdi>: Unsupported mount device: /dev/sr0 /media/cdrom0 [dissect.target.target]

All filesystems (root, boot and home) are mounted at the correct places. I assume this should be the case with all supported filesystems?

Edit: acquire also does not mount the partitions and misses the files in home for example

Schamper commented 1 year ago

The expected behaviour is indeed that all supported filesystems from /etc/fstab are mounted at the correct directories. However, with the changes from #237 it should mount every detected filesystem at filesystems/<name or fs_#>. Can you verify if that is also not the case now?

Can you share some details on the Debian VM for debugging purposes? I.e. what version of Debian, is it a clean install, how are the volumes configured, or is anything else noteworthy configured such as LUKS? That way we can try to reproduce.

Reopening this for now.

mischw commented 1 year ago

I hope I did not misunderstand. The partitions are mounted fine in filesystems/ but not in fs/. The weird thing to me is that they only show up in fs/ too if they are formatted as xfs. For the VMs I used the debian-11.6.0-amd64-netinst.iso and did a quick offline install accepting the default values, except I did the partitioning manually (using the installer).

5.vdi, using ext4 and ext2:

target-fs /mnt/c/WSL/debpart/5.vdi cat /etc/fstab
2023-06-21T07:00:11.293471Z [warning  ] <Target /mnt/c/WSL/debpart/5.vdi>: Unsupported mount device: /dev/sr0 /media/cdrom0 [dissect.target.target]
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# systemd generates mount units based on this file, see systemd.mount(5).
# Please run 'systemctl daemon-reload' after making changes here.
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# / was on /dev/sda2 during installation
UUID=2eb9a171-1c43-4f10-bf45-fa302df55d46 /               ext4    errors=remount-ro 0       1
# /boot was on /dev/sda1 during installation
UUID=dee9d810-ba16-4a67-9c43-a50825ffcf44 /boot           ext2    defaults        0       2
# /home was on /dev/sda3 during installation
UUID=217f0e13-9bd4-4000-85ad-c810f7cea9ab /home           ext4    defaults        0       2
/dev/sr0        /media/cdrom0   udf,iso9660 user,noauto     0       0
target-mount /mnt/c/WSL/debpart/5.vdi /mnt/TEMP
2023-06-21T07:02:23.985628Z [warning  ] <Target /mnt/c/WSL/debpart/5.vdi>: Unsupported mount device: /dev/sr0 /media/cdrom0 [dissect.target.target]
Mounting to /mnt/TEMP with options: allow_other,ro

Here they are accessible:

ls -la /mnt/TEMP/filesystems/part_*/
/mnt/TEMP/filesystems/part_00100000/:
total 0
drwxr-xr-x 4 root root     4096 Jun 20 10:29 .
d--------- 0 root root        0 Jan  1  1970 ..
-rw-r--r-- 1 root root   236452 Dec 13  2022 config-5.10.0-20-amd64
drwxr-xr-x 5 root root     4096 Jun 20 10:28 grub
-rw-r--r-- 1 root root 28990988 Jun 20 10:29 initrd.img-5.10.0-20-amd64
drwx------ 2 root root    16384 Jun 20 10:21 lost+found
-rw-r--r-- 1 root root       83 Dec 13  2022 System.map-5.10.0-20-amd64
-rw-r--r-- 1 root root  7008928 Dec 13  2022 vmlinuz-5.10.0-20-amd64

/mnt/TEMP/filesystems/part_28fb00000/:
total 0
drwxr-xr-x 4 root   root    4096 Jun 20 10:28 .
d--------- 0 root   root       0 Jan  1  1970 ..
drwx------ 2 root   root   16384 Jun 20 10:21 lost+found
drwxr-xr-x 2 myuser myuser  4096 Jun 20 10:30 myuser

/mnt/TEMP/filesystems/part_3ba00000/:
ls: cannot access '/mnt/TEMP/filesystems/part_3ba00000/vmlinuz.old': Input/output error
ls: cannot access '/mnt/TEMP/filesystems/part_3ba00000/initrd.img.old': Input/output error
ls: cannot access '/mnt/TEMP/filesystems/part_3ba00000/vmlinuz': Input/output error
ls: cannot access '/mnt/TEMP/filesystems/part_3ba00000/initrd.img': Input/output error
total 0
drwxr-xr-x 18 root root  4096 Jun 20 10:24 .
d---------  0 root root     0 Jan  1  1970 ..
drwxr-xr-x  2 root root 20480 Jun 20 10:28 bin
drwxr-xr-x  2 root root  4096 Jun 20 10:21 boot
drwxr-xr-x  4 root root  4096 Jun 20 10:21 dev
drwxr-xr-x 60 root root  4096 Jun 20 10:29 etc
drwxr-xr-x  2 root root  4096 Jun 20 10:21 home
??????????  ? ?    ?        ?            ? initrd.img
??????????  ? ?    ?        ?            ? initrd.img.old
drwxr-xr-x 45 root root  4096 Jun 20 10:28 lib
drwxr-xr-x  2 root root  4096 Jun 20 10:21 lib32
drwxr-xr-x  2 root root  4096 Jun 20 10:21 lib64
drwxr-xr-x  2 root root  4096 Jun 20 10:21 libx32
drwx------  2 root root 16384 Jun 20 10:21 lost+found
drwxr-xr-x  3 root root  4096 Jun 20 10:21 media
drwxr-xr-x  2 root root  4096 Jun 20 10:21 mnt
drwxr-xr-x  2 root root  4096 Jun 20 10:21 opt
drwxr-xr-x  2 root root  4096 Dec  9  2022 proc
drwx------  2 root root  4096 Jun 20 10:21 root
drwxr-xr-x  2 root root  4096 Jun 20 10:29 run
drwxr-xr-x  2 root root  4096 Jun 20 10:29 sbin
drwxr-xr-x  2 root root  4096 Jun 20 10:21 srv
drwxr-xr-x  2 root root  4096 Dec  9  2022 sys
drwxrwxrwt  7 root root  4096 Jun 20 10:30 tmp
drwxr-xr-x 14 root root  4096 Jun 20 10:21 usr
drwxr-xr-x 11 root root  4096 Jun 20 10:21 var
??????????  ? ?    ?        ?            ? vmlinuz
??????????  ? ?    ?        ?            ? vmlinuz.old

But here they don't show up. This is the part that is strange to me:

ls -la /mnt/TEMP/fs/home
total 0
drwxr-xr-x  2 root root 4096 Jun 20 10:21 .
drwxr-xr-x 18 root root 4096 Jun 20 10:24 ..

6.vdi, using only xfs

target-fs /mnt/c/WSL/debpart/6.vdi cat /etc/fstab
2023-06-21T07:00:39.374681Z [warning  ] <Target /mnt/c/WSL/debpart/6.vdi>: Unsupported mount device: /dev/sr0 /media/cdrom0 [dissect.target.target]
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# systemd generates mount units based on this file, see systemd.mount(5).
# Please run 'systemctl daemon-reload' after making changes here.
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# / was on /dev/sda2 during installation
UUID=1df44dcb-a35a-4b80-b361-9fd880a25657 /               xfs     defaults        0       0
# /boot was on /dev/sda1 during installation
UUID=29aacf3e-8478-4e2c-a845-7205ecfff4d8 /boot           xfs     defaults        0       0
# /home was on /dev/sda3 during installation
UUID=38b280a7-0c2e-42b7-824d-11589467d482 /home           xfs     defaults        0       0
/dev/sr0        /media/cdrom0   udf,iso9660 user,noauto     0       0
target-mount /mnt/c/WSL/debpart/6.vdi /mnt/TEMP
2023-06-21T07:06:35.257018Z [warning  ] <Target /mnt/c/WSL/debpart/6.vdi>: Unsupported mount device: /dev/sr0 /media/cdrom0 [dissect.target.target]
Mounting to /mnt/TEMP with options: allow_other,ro

Also fine, as expected:

ls -la /mnt/TEMP/filesystems/part_*/
/mnt/TEMP/filesystems/part_00100000/:
total 0
drwxr-xr-x 3 root root      147 Jun 20 10:44 .
d--------- 0 root root        0 Jan  1  1970 ..
-rw-r--r-- 1 root root   236452 Dec 13  2022 config-5.10.0-20-amd64
drwxr-xr-x 5 root root       98 Jun 20 10:44 grub
-rw-r--r-- 1 root root 29240358 Jun 20 10:44 initrd.img-5.10.0-20-amd64
-rw-r--r-- 1 root root       83 Dec 13  2022 System.map-5.10.0-20-amd64
-rw-r--r-- 1 root root  7008928 Dec 13  2022 vmlinuz-5.10.0-20-amd64

/mnt/TEMP/filesystems/part_28fb00000/:
total 0
drwxr-xr-x 3 root   root   20 Jun 20 10:44 .
d--------- 0 root   root    0 Jan  1  1970 ..
drwxr-xr-x 2 myuser myuser 78 Jun 20 10:52 myuser

/mnt/TEMP/filesystems/part_3ba00000/:
ls: cannot access '/mnt/TEMP/filesystems/part_3ba00000/vmlinuz.old': Input/output error
ls: cannot access '/mnt/TEMP/filesystems/part_3ba00000/initrd.img.old': Input/output error
ls: cannot access '/mnt/TEMP/filesystems/part_3ba00000/vmlinuz': Input/output error
ls: cannot access '/mnt/TEMP/filesystems/part_3ba00000/initrd.img': Input/output error
total 0
drwxr-xr-x 17 root root   325 Jun 20 10:38 .
d---------  0 root root     0 Jan  1  1970 ..
drwxr-xr-x  2 root root 12288 Jun 20 10:44 bin
drwxr-xr-x  2 root root     6 Jun 20 10:35 boot
drwxr-xr-x  4 root root   182 Jun 20 10:35 dev
drwxr-xr-x 60 root root  4096 Jun 20 10:45 etc
drwxr-xr-x  2 root root     6 Jun 20 10:35 home
??????????  ? ?    ?        ?            ? initrd.img
??????????  ? ?    ?        ?            ? initrd.img.old
drwxr-xr-x 46 root root  4096 Jun 20 10:44 lib
drwxr-xr-x  2 root root     6 Jun 20 10:35 lib32
drwxr-xr-x  2 root root    34 Jun 20 10:36 lib64
drwxr-xr-x  2 root root     6 Jun 20 10:35 libx32
drwxr-xr-x  3 root root    33 Jun 20 10:35 media
drwxr-xr-x  2 root root     6 Jun 20 10:36 mnt
drwxr-xr-x  2 root root     6 Jun 20 10:36 opt
drwxr-xr-x  2 root root     6 Dec  9  2022 proc
drwx------  2 root root    37 Jun 20 10:36 root
drwxr-xr-x  2 root root     6 Jun 20 10:45 run
drwxr-xr-x  2 root root  8192 Jun 20 10:45 sbin
drwxr-xr-x  2 root root     6 Jun 20 10:36 srv
drwxr-xr-x  2 root root     6 Dec  9  2022 sys
drwxrwxrwt  7 root root    93 Jun 20 10:52 tmp
drwxr-xr-x 14 root root   160 Jun 20 10:36 usr
drwxr-xr-x 11 root root   139 Jun 20 10:36 var
??????????  ? ?    ?        ?            ? vmlinuz
??????????  ? ?    ?        ?            ? vmlinuz.old

And even in fs/ they show up. See home for example:

ls -la /mnt/TEMP/fs/home
total 0
drwxr-xr-x  2 root   root     6 Jun 20 10:35 .
drwxr-xr-x 17 root   root   325 Jun 20 10:38 ..
drwxr-xr-x  2 myuser myuser  78 Jun 20 10:52 myuser

Besides some Input/output error (Is this expected?) which happens in both cases there are no errors. Let me know if I can provide more info.

Edit: If you want to try it yourself, I get the same behavior where no mounting under fs/ is happening with this image: https://sourceforge.net/projects/osboxes/files/v/vm/14-D-bn/11/Server/64bit.7z/download

Schamper commented 1 year ago

Thanks for the detailed information, you understood correctly! The link you provided indeed also allows me to reproduce so I'll look into this!

Schamper commented 1 year ago

Managed to spot it quickly now, once again an argument against using strings vs enums/constants... :) Perhaps we should change those to proper enums down the line.

mischw commented 1 year ago

Thank you for the quick fix! I can confirm it now works as expected :)