MFT not being parsed using default log2timeline parser arguments on an NTFS disk image

[john-corcoran]

Plaso version:

20170930

Operating system Plaso is running on:

Windows 10 x64

Installation method:

Windows packaged release at https://github.com/log2timeline/plaso/releases

Description of problem:

No MFT events are being generated when a Windows disk image is being processed with default log2timeline arguments, as follows: log2timeline.exe --logfile log.txt -d output.db image.E01

MFT events are however generated when running with the mft parser explicitly selected: log2timeline.exe --logfile log.txt --parsers mft -d output.db image.E01

MFT events are also generated if the MFT files are copied to a folder: log2timeline.exe --logfile log.txt -d output.db /folderpath/

Perhaps this is normal behaviour and filestat events replace MFT parsing on default settings for Windows disk images?

Debug output/tracebacks:

log2timeline.exe --logfile log.txt -d output.db image.E01 For this run, the $MFT and $MFTMirr files are identified and scheduled tasks are queued, but the tasks never enter processing: merge and task completion occurs only 14 seconds after the task is originally queued.

log2timeline.exe --logfile log.txt --parsers mft -d output.db image.E01 For this run, the $MFT and $MFTMirr are queued and also enter processing, before being merged and completed a number of hours later.

Source data:

Windows 7 disk image (a compressed E01 file) with one NTFS partition.

[Onager]

This is because the MFT parser isn't enabled by default when Plaso detects a Windows operating system. If you enable the win7_slow parser preset: https://github.com/log2timeline/plaso/blob/master/plaso/parsers/presets.py the MFT parser will be enabled.

This is because the MFT parser takes a long time to run, and we need to add some extra features to increase its usefulness: https://github.com/log2timeline/plaso/issues/316

[EricZimmerman]

you should look at MFTECmd for processing MFTs. it handles most, regardless of size in just a few seconds to a minute or so for really large (1GB+) MFTs =)

[joachimmetz]

@EricZimmerman very nice that you are pitching your own tools here. A couple of considerations:

• Can you run MFTECmd directly on a BDE encrypted volume?
• Does is correctly handle NTFS sequence numbers?
• Does it correctly handle full paths, not blindly assuming the name of the parent is the actual original path (e.g. hard links)?

If it uses the same naive approach and blind assumptions as most of the other MFT parsers out there, from a forensics point of view I don't think speed is an improvement.

Also another bad advice from your side:

Any anti-virus hits are false positives and the warnings can be ignored.

Source: https://ericzimmerman.github.io/#!index.md

[EricZimmerman]

people need MFT processing, and plasos is slow from what was stated above and not run by default, so why not mention another tool people can use that is open source, free, and works well?

Rather than just make assumptions, i would suggest running MFTECmd and maybe providing feedback if you see any deficiencies in its design or implementation.

it does not make naive approaches to anything that I am aware of. it actually gets deleted files right (as best as i can tell) and does secondary checks to make sure a deleted file isn't reassociated with a file for example, etc.

• no, can't run it against BitLocker directly. it parses the $MFT file directly after $MFT is extracted from a volume
• What do you mean, handles NTFS sequence numbers? in what regard?
• it handles hard links just fine and resolves the proper parent directory based on MFT entry and sequence number, taking into account how FILE records change when they are deleted.
• it does not rely on attribute lists to find associated records, as this is a faulty approach when attribute lists are non-resident. i have only seen one other parser that doesn't make this mistake when dealing with a loose MFT. how does your code handle attribute lists as it relates to records that span more than one FILE entry when you only have an $MFT? i realize having a full disk image negates this, but then again, processing a loose MFT has its advantages from a speed perspective vs requiring having a full disk.

i found and reported significant bugs in other tools, like istat, etc, during this project as well.

They say speed is fine, but accuracy is final. MFTECmd fits the bill in both regards. it processes MFTs in a minute or so that other tools literally take an hour to process, if they will even be able to process them at all.

i dont see how ignoring false positives is a security risk. is something malware because Norton said so? I dont think so.

[joachimmetz]

I'll continue with you this conversation off thread, seeing that you are hijacking this issue now multiple times for your own purposes.

[joachimmetz]

For context about the "accuracy" of MFTECmd https://github.com/EricZimmerman/MFTECmd/issues/5

Another thing to note is that MFTECmd removes timestamp precision when writing to bodyfile.