Finding files forever

[klasse76]

Hi

I have imported a E01 into Griffeye Analyze whihc is using Sleuthkit for extracting images into the software. It found about 40 times the amount of files that I have in my image according to EnCase.

I then ran the same E01 in Autopsy and the same thing happened. It ran and ran for days and nothing happened The image contains illegal files so I can’t send it to you. I have installed autopsy and imported the E01 image, selecting all options except email parsing, android, keywords. It has been running now for days and is still going, I have no idea how much longer it is going to take.

I gave up on the Analyze job at 155 million files – it was still going but I had to restart my computer. The image contains 4 million files according to EnCase.

What could be the problem?

[bcarrier]

Hello. Do you know what file system type it is? FAT?

When you added it to Autopsy, did the "Add Data Source" wizard panel ever go away and you could see the tree and such to navigate around? There are two phases to adding a Data Source in Autopsy. One is when The Sleuth Kit is used to enumerate all of the files (and that can be slow for FAT file systems) and the second is when the files are run through the pipelines of ingest modules. The tree on the left doesn't appear until the files have been enumerated.

[klasse76]

Hi

It is an external storage drive (ie there is no OS), which has 1.8TB formatted as HFSX, and a 200MB FAT32 partition. It is a 2TB drive. It is an E01 image. Sleuth kit never got to the point where there was a tree visible – I got the Configure ingest module window, clicked next and then got a screen that says processing data source and adding it to a local database. *this process may take some time for large data sources. After 24 hours it was still on this same screen though the file path shown in the lower window is changing. I then accidentally pressed something and it stopped (probably I accidentally pressed the cancel button). I will run it again over the weekend and see how far it gets. Screen shots attached.

[klasse76]

Well autopsy has been running all weekend and is still going. What seems to be taking all its time is a large number of folders named Dir_123456 etc within folder called .HFS+ Private Directory data. Posts on the internet seem to suggest that this is related to Apple’s time machine. The contents of these folders is not shown at all in Encase – it shows folders and subfolders– but not files

[klasse76]

Do you have any new input in this issue or any other information that I have to give you to resolve it? @bcarrier

[klasse76]

Is anything happening with this issue or has it gone dead?

[klasse76]

Do you have any idea what this could be? Can't proceed in the case before I get this resolved? @bcarrier

[bcarrier]

Hi @klasse76, I haven't had a chance to recreate this yet. Those folders are associated with hard links in HFS+. EnCase is showing one of the folders, but not the 2nd (perhaps because the folder name has lots of NULL characters in it). It could also be the unnamed folder that is in the EnCase screen shot.

Does the Autopsy progress always show it being in one of the "private folder"? My memory of the HFS hard links is that those folders store the actual file content and other directory structures have references back into them. Apple does a lot to hide these folders from normal users.

I could imagine that if the image has a lot of hard links to the same files (from a backup) that Autopsy/TSK would process each file as a new file and it could take a long time because it doesn't realize it already saw that content.

But, I'm surprised it is taking this long at this stage in the process. I would expect it to take a long time to run all of the files down the Autopsy pipeline, but that it should be able to add them all to the DB.

Maybe the TSK implementation of resolving hard links is inefficient. We'll try to schedule some time to look at this.

[bcarrier]

Hi @klasse76. We made a test image here and are working on this. I loaded the test image into EnCase 6 and noticed that it did not show the file content associated with any of the hard links. It just showed the name of the link with an exclamation point next to it. The only content for the hard links was in a folder that had no name and the files were named 'iNodeX'.

Is that the same behavior that you are seeing in newer versions of EnCase? I can send you a small image to test with if you can't easily figure it out with your bigger data set.

At this point we are going to look into:

• Why is it taking so long for us to resolve the hard links (NOTE that EnCase 6 at least is not trying to resolve the links and therefor is going to be much faster. But, there are probably optimizations we can make so that it doesn't take days!)
• Making sure that we don't process the same file content multiple times. For most images, there are only a handful of hard links and it isn't an issue. With a time machine backup though, they are nearly all hard links.

[klasse76]

Hi Brian, Been on vacation. Do you have a small test image that would be great and I can see if it can be resolved more quickly thank you.

Klas

[APriestman]

Hi Klas, here is the small test image we made to look at hard links, and a screenshot of how it looks in Autopsy. In Autopsy we see the correct data in both the iNode files and the original versions (for example, "output/A/a/000_link.txt").

hfs_hardlink_test_small.zip