Error loading EWF file when similar named files exist in folder

[mischw]

There is an issue with loading EWF files when there are other similar named files in the same folder which are not part of the image. Example:

$ ls
test.e01  test.e02  test.e03  test.txt

$ target-info test.e01
2024-07-17T09:44:34.963276Z [error    ] test.e01: Failed to load target with loader RawLoader('test.e01') [dissect.target.target]
Traceback (most recent call last):
  File "/root/test/.venv/lib/python3.12/site-packages/dissect/target/container.py", line 237, in open
    return container(item, \**args, \**\\*kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/test/.venv/lib/python3.12/site-packages/dissect/target/helpers/lazy.py", line 60, in __call__
    return self.realattr(\**args, \**\\*kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/test/.venv/lib/python3.12/site-packages/dissect/target/containers/ewf.py", line 22, in __init__
    self.ewf = EWF(find_files(fhs[0]))
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/test/.venv/lib/python3.12/site-packages/dissect/evidence/ewf.py", line 168, in __init__
    last_table = self.open_segment(len(self.fh) - 1).tables[-1]
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/test/.venv/lib/python3.12/site-packages/dissect/evidence/ewf.py", line 197, in open_segment
    segment = Segment(self, fh)
              ^^^^^^^^^^^^^^^^^
  File "/root/test/.venv/lib/python3.12/site-packages/dissect/evidence/ewf.py", line 255, in __init__
    self.ewfheader = c_ewf.EWFHeader(fh)
                     ^^^^^^^^^^^^^^^^^^^
  File "/root/test/.venv/lib/python3.12/site-packages/dissect/cstruct/types/structure.py", line 72, in __call__
    return super().__call__(\**args, \**\\*kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/test/.venv/lib/python3.12/site-packages/dissect/cstruct/types/base.py", line 40, in __call__
    return cls._read(stream)
           ^^^^^^^^^^^^^^^^^
  File "<compiled EWFHeader._read>", line 7, in _read
EOFError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/root/test/.venv/lib/python3.12/site-packages/dissect/target/target.py", line 419, in _load
    ldr.map(target)
  File "/root/test/.venv/lib/python3.12/site-packages/dissect/target/loaders/raw.py", line 23, in map
    target.disks.add(container.open(self.path))
                     ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/test/.venv/lib/python3.12/site-packages/dissect/target/container.py", line 242, in open
    raise ContainerError(f"Failed to open container {item}", cause=e)
dissect.target.exceptions.ContainerError: Failed to open container test.e01

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/root/test/.venv/lib/python3.12/site-packages/dissect/target/target.py", line 298, in open_all
    target = cls._load(sub_entry, ldr)
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/test/.venv/lib/python3.12/site-packages/dissect/target/target.py", line 424, in _load
    raise TargetError(f"Failed to load target: {path}", cause=e)
dissect.target.exceptions.TargetError: Failed to load target: test.e01
2024-07-17T09:44:34.964374Z [error    ] Failed to find any loader for targets: ['test.e01'] [dissect.target.tools.info]

Here it fails to load the image because of the test.txt file, which is not actually part of the image. It does think so because the glob in this function for finding the EWF files may include many other file extensions. For example renaming the test.txt to test.png results in the same error. When deleting the file test.txt it works fine again. Unfortunately a txt file is often accompanying the image to provide further metadata.

To resolve this I think dissect should check if the files found are actually part of the image. The easiest approach might be to check if all files found start with the EWF magic bytes ("EVF") and discard the ones which do not. Since the Magic Bytes are already checked here this could be done by showing a warning instead of raising an exception.

[Horofic]

Hello @mischw, thank you for raising this issue! Just to make it clear for us; Is this regular behaviour for acquisition tools?

I recall that most of these tools create a test.e01.txt file instead of a test.txt file, since the .txt extension can technically occur during large acquisition. Would you be able to tell what tool produced this output file?

In the meantime we will try and think of a possible solution.

[Schamper]

This looks like a bug in the way we open EWF segments. In the initial iteration of the segments, it will be properly skipped: https://github.com/fox-it/dissect.evidence/blob/852ced41dea9fc0a506f881847606499214c3a37/dissect/evidence/ewf.py#L143-L147

But then a few lines later we always assume the last file-like object is a valid segment: https://github.com/fox-it/dissect.evidence/blob/852ced41dea9fc0a506f881847606499214c3a37/dissect/evidence/ewf.py#L168

The (in)valid segment indices should be recorded and also checked in open_segment, or cleaned up from the list of file-like objects.

[mischw]

@Horofic Unfortunately X-Ways Forensics does write a test.txt instead of a test.e01.txt when acquiring an EWF image.

@Schamper Now that I took a closer look I think you are right. It does skip the txt file initially and then fails at the last call to open_segment. Might make sense to remove them early in the process so one does not need to worry about bad segments further down the line.