search

sleuthkit / autopsy

Autopsy® is a digital forensics platform and graphical interface to The Sleuth Kit® and other digital forensics tools. It can be used by law enforcement, military, and corporate examiners to investigate what happened on a computer. You can even use it to recover photos from your camera's memory card.
http://www.sleuthkit.org/autopsy/
2.31k stars 588 forks source link

.TAR files not extracted with ingest module #6924

Open TroySchnack opened 3 years ago

TroySchnack commented 3 years ago

Embedded file extraction works on ZIPs, GZs and other, but does not work on TAR files. This is common in iOS and other device imaging. Would be nice it could expand TAR files for additional ingest modules, searching and more.

betsy-art commented 3 years ago

I'm having the same issue. I have tried countless methods to get the TARs to ingest and parse to no avail.

bcarrier commented 3 years ago

If the mime type is detected as "application/x-tar", then the module should process it.

What is the file type of the files you have?

TroySchnack commented 3 years ago

iOS extractions - samples can be found from https://thebinaryhick.blog - each time I have to manually decompress with 7-Zip or other before using other Ingest modules.

betsy-art commented 3 years ago

I have multiple Android logical images, from multiple sources (Magnet, OSF, and MPE). The image contents are contained with the TAR. I've tried decompressing the TAR into just a folder directory as well. I'm unable to ingest any of the data to be recognized in Autopsy. I've also installed the Python plugins for parsing the .db within the TAR.

bcarrier commented 3 years ago

I just downloaded an iOS tar file from the site Troy mentioned. Added the TAR as a "Logical File" and then enabled three modules for the test:

Embedded File Extractor extracted files from the TAR: image

And then hung at 240k out of 345k files. The thread is at:

"IM-file-ingest-0" Id=39 TIMED_WAITING on java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@2a2f928 at sun.misc.Unsafe.park(Native Method)

We'll take a look at this. But, my test at least is that we do support TAR files, but something is holding this up. Could be because it's got 350k files in it....

TroySchnack commented 3 years ago

Thanks so much for investigating and validating the issue.

betsy-art commented 3 years ago

Thank you for your help! Using the above three modules, I tested my Android TAR, using the Android module, rather than IOS. The TAR has less than 4,000 files. However, I'm experiencing the same issues as well.

markmckinnon commented 3 years ago

My suggestion which may prove quicker for both because of the number of files is to create a VHD (format ntfs file system) bigger then the tar file and then mount it. You can use 7zip to extract to drive letter the vhd is mounted to. Once complete then unmount it and then use the VHD as the data source (image). A few benefits of doing this will be faster processing time for many things since you are using the files from the image instead of a derived file, modified date/time will be the same as the file when it was tared up, the birth and last access will be the date and time when you untared it, with a derived file the dates/times will not be there. I am sure there are a few more benefits as well.

TroySchnack commented 3 years ago

My suggestion which may prove quicker for both because of the number of files is to create a VHD (format ntfs file system) bigger then the tar file and then mount it. You can use 7zip to extract to drive letter the vhd is mounted to. Once complete then unmount it and then use the VHD as the data source (image). A few benefits of doing this will be faster processing time for many things since you are using the files from the image instead of a derived file, modified date/time will be the same as the file when it was tared up, the birth and last access will be the date and time when you untared it, with a derived file the dates/times will not be there. I am sure there are a few more benefits as well.

Great tip Mark. Thanks

betsy-art commented 3 years ago

Thank you, Mark! I'll give it a try and revert.

bcarrier commented 3 years ago

As an update for me, it's still going on my system. it got past that 240k point, but is going slow. A few hours in and it's still adding files back into the DB.

betsy-art commented 3 years ago

The VHD tip yielded the same results…Here is what I’ve tried thus far.

  1. Magnet Android image > Direct ingest of the TAR file
  2. Decompress the TAR with 7ZIP, to a folder on a my local drive, ingest logical folder/files
  3. Re-image with OSF output VHD, direct ingest of the VHD
  4. Create 5GB NTFS VHD, mount on my local system, extract the contents of the TAR to the VHD, and ingest the VHD.
  5. Processed the TAR in FTK, exported out the contents, ingested as a logical folder.
  6. After research, I tried this as well (LINK: Forensic analysis of an Android logical image with Autopsy | Digital Forensics | Computer Forensics | Blog)

I have the same results with all of the above.

markmckinnon commented 3 years ago

@betsy-art, if you can shoot me an email, we can take this offline and see what is going on. My email is on my GitHub profile.