search

sepinf-inc / IPED

IPED Digital Forensic Tool. It is an open source software that can be used to process and analyze digital evidence, often seized at crime scenes by law enforcement or in a corporate investigation by private examiners.
Other
962 stars 219 forks source link

NPE in SleuthkitClient when generating report with a virtual disk #1870

Closed aberenguel closed 1 year ago

aberenguel commented 1 year ago

Using master branch (f6eae92b9).

The exception is:

java.lang.Exception: Worker-0 Error while processing /item02.E01/vol_vol7/Windows/System32/config/SECURITY (32768bytes)
    at iped.engine.core.Worker.process(Worker.java:186)
    at iped.engine.core.Worker.run(Worker.java:265)
Caused by: java.lang.NullPointerException
    at iped.engine.sleuthkit.SleuthkitClient.get(SleuthkitClient.java:127)
    at iped.engine.sleuthkit.SleuthkitInputStreamFactory.getSeekableInputStream(SleuthkitInputStreamFactory.java:124)
    at iped.engine.data.Item.getSeekableInputStream(Item.java:562)
    at iped.engine.data.Item.getBufferedInputStream(Item.java:281)
    at iped.engine.data.Item.getTikaStream(Item.java:647)
    at iped.engine.task.ParsingTask.safeProcess(ParsingTask.java:380)
    at iped.engine.task.ParsingTask.process(ParsingTask.java:329)
    at iped.engine.task.AbstractTask.processMonitorTimeout(AbstractTask.java:277)
    at iped.engine.task.AbstractTask.processAndSendToNextTask(AbstractTask.java:192)
    at iped.engine.task.AbstractTask.sendToNextTask(AbstractTask.java:225)
    at iped.engine.task.AbstractTask.processAndSendToNextTask(AbstractTask.java:205)
    at iped.engine.task.AbstractTask.sendToNextTask(AbstractTask.java:225)
    at iped.engine.task.AbstractTask.processAndSendToNextTask(AbstractTask.java:205)
    at iped.engine.task.AbstractTask.sendToNextTask(AbstractTask.java:225)
    at iped.engine.task.AbstractTask.processAndSendToNextTask(AbstractTask.java:205)
    at iped.engine.task.AbstractTask.sendToNextTask(AbstractTask.java:225)
    at iped.engine.task.AbstractTask.processAndSendToNextTask(AbstractTask.java:205)
    at iped.engine.task.AbstractTask.sendToNextTask(AbstractTask.java:225)
    at iped.engine.task.AbstractTask.processAndSendToNextTask(AbstractTask.java:205)
    at iped.engine.task.AbstractTask.sendToNextTask(AbstractTask.java:225)
    at iped.engine.task.AbstractTask.processAndSendToNextTask(AbstractTask.java:205)
    at iped.engine.task.AbstractTask.sendToNextTask(AbstractTask.java:225)
    at iped.engine.task.AbstractTask.processAndSendToNextTask(AbstractTask.java:205)
    at iped.engine.task.AbstractTask.sendToNextTask(AbstractTask.java:225)
    at iped.engine.task.AbstractTask.processAndSendToNextTask(AbstractTask.java:205)
    at iped.engine.task.AbstractTask.sendToNextTask(AbstractTask.java:225)
    at iped.engine.task.AbstractTask.processAndSendToNextTask(AbstractTask.java:205)
    at iped.engine.task.AbstractTask.sendToNextTask(AbstractTask.java:225)
    at iped.engine.task.AbstractTask.processAndSendToNextTask(AbstractTask.java:205)
    at iped.engine.task.AbstractTask.sendToNextTask(AbstractTask.java:225)
    at iped.engine.task.AbstractTask.processAndSendToNextTask(AbstractTask.java:205)
    at iped.engine.task.AbstractTask.sendToNextTask(AbstractTask.java:225)
    at iped.engine.task.AbstractTask.processAndSendToNextTask(AbstractTask.java:205)
    at iped.engine.core.Worker.process(Worker.java:177)
    ... 1 more

Debugging the code, I checkout out none of SleuthkitClient instances were created when the method SleuthkitClient.get() was called. Maybe this happened due to slowness in my computer.

I created a patch that uses conditional variables in to be verified in SleuthkitClient.get() and signalized in initSleuthkitServers().

Pushing soon.

lfcnassif commented 1 year ago

Thanks for reporting and debugging @aberenguel! Could you detail how the report was being created? Not sure, but I think SleuthkitClient.get() shouldn't be called when creating reports because the first clause of if below should pass: https://github.com/sepinf-inc/IPED/blob/f6eae92b9dbab410d873efc6427b1e101c61b30b/iped-engine/src/main/java/iped/engine/sleuthkit/SleuthkitInputStreamFactory.java#L121

Unless you are processing a new evidence together with report creation from command line, are you?

aberenguel commented 1 year ago

I'm using the GUI to generate the report. I'll try to check why the flow is not entering in the "if".

lfcnassif commented 1 year ago

Maybe an embedded disk was processed during report generation, that would call SleuthkitReader.read(...) and populate SleuthkitReader.sleuthCase. Is there any virtual disk (or other kind of embedded disk) in your bookmarks included in the report? If yes, I think we should disable embedded disk expansion in report creation, like we do for ordinary containers...

aberenguel commented 1 year ago

Maybe an embedded disk was processed during report generation, that would call SleuthkitReader.read(...) and populate SleuthkitReader.sleuthCase. Is there any virtual disk (or other kind of embedded disk) in your bookmarks included in the report? If yes, I think we should disable embedded disk expansion in report creation, like we do for ordinary containers...

I think that is the answer. I my case I have two Android emulators related to some VMDK files. I exported the VMDK files, processed in PA and appended the UFDR in the case. My idea was to add the VMDK in the report but without expand it.

lfcnassif commented 1 year ago

Great! Try to turn processEmbeddedDisks off in IPEDConfig.txt into your case and to generate the report again, I think it should also fix the issue (without the patch) and it won't expand the VMDKs into the report.

aberenguel commented 1 year ago

I just analyzed the VMDK files again. In the original processing, there were 6 files with extension vmdk.

One of them was not recognized by Sleuthkit. image

aberenguel commented 1 year ago

The above VMDK file was detected like that by file command in Linux

./MEmu/MemuHyperv VMs/MEmu/MEmu71-2022012700003FFF-disk2.vmdk: VMware4 disk image
aberenguel commented 1 year ago

Is there any way to disable the split of large binaries? I've seen enableSplitLargeBinary = false in profiles triage and fastmode, but it seems it is not being used.

lfcnassif commented 1 year ago

Is there any way to disable the split of large binaries? I've seen enableSplitLargeBinary = false in profiles triage and fastmode, but it seems it is not being used.

It is not possible anymore, it is always enabled to avoid #1281. We should remove the enableSplitLargeBinary option from all places if it still exists...

lfcnassif commented 1 year ago

The above VMDK file was detected like that by file command in Linux

Is it a single segment VMDK? Maybe it is just a VMDK part and IPED wasn't able to detect all its parts before exporting them and sending to TSK for decoding...

lfcnassif commented 1 year ago

I've seen enableSplitLargeBinary = false in profiles triage and fastmode, but it seems it is not being used.

Just removed them in commit af66ce910ca46da7d54352b8a7cab54e22b825aa