search

sleuthkit / autopsy

Autopsy® is a digital forensics platform and graphical interface to The Sleuth Kit® and other digital forensics tools. It can be used by law enforcement, military, and corporate examiners to investigate what happened on a computer. You can even use it to recover photos from your camera's memory card.
http://www.sleuthkit.org/autopsy/
2.41k stars 597 forks source link

Recent Activity module fails to complete on Linux #5802

Closed mishaturnbull closed 4 years ago

mishaturnbull commented 4 years ago

I'm having an issue attempting to get through the training course which I believe is located in the Recent Activity ingest module. The symptoms present as the module not completing successfully, notably not analyzing registry hives.

In the ingest inbox, I get a mail containing the following text:

Errors encountered during analysis:
    * Recent Activity had errors -- see log
    * Recent Activity had errors -- see log

The module gives me accounts, recent documents, recycle bin, web bookmarks, web cache, web cookies, web downloads, form autofill, web history and search. I am unable to view registry analysis, such as USB device history.

Full messages.log file is at https://pastebin.com/24fnfFUc for brevity.

If it helps, the data source is laptop.e01 in the training module. I'm hesitant to post a link here, since the course is not (usually) free, but can do if needed.

I did find that a recent issue #5801 seems to share chunks of the error log, notably the missing guava-17.0.jar file. However, I'm only running on a 1920x1080 screen so I don't believe this is the same issue. Like tonybounty, I also run other javafx applications without issue. Other than this, two days of Googling for topics such as "autopsy recent activity had errors" turned up nothing.

Unfortunately, I don't currently have access to a Windows platform to test on, so I haven't been able to replicate the issue there. Two reinstalls of Autopsy reproduce the issue for me, however.

markmckinnon commented 4 years ago

You may be running into this problem where regripper was not executing correctly in Linux. You can view more information here https://github.com/sleuthkit/autopsy/pull/5636

bcarrier commented 4 years ago

@mishaturnbull We've received several comments about this on the training image, but have not been able to recreate it. One person reported that when they closed the case and opened it again, that the results were shown. If so, this suggests some kind of UI refresh issue.

Can you test this?

mishaturnbull commented 4 years ago

@markmckinnon That seems to address the correct issue, but I'm unable to test effectively as I can't get the current git version set up. I've built and installed the latest commit from https://github.com/sleuthkit/sleuthkit from source and verified the file /usr/local/java/sleuthkit-4.9.0.jar exists, but still am unable to complete the Autopsy install.

# sh unix_setup.sh  # running as root here, shouldn't get any permissions errors.  happens as normal user too
---------------------------------------------
Checking prerequisites and preparing Autopsy:
---------------------------------------------
Checking for PhotoRec...found in /usr/bin
Checking for Java...found in /usr/lib/jvm/bellsoft-java8-full-amd64
Checking for Sleuth Kit Java bindings...found in /usr/local/share/java
Copying sleuthkit-4.9.0.jar into the Autopsy directory...cp: cannot create regular file '/home/misha/Downloads/autopsy-git/autopsy/modules/ext/sleuthkit-postgresql-4.9.0.jar': No such file or directory
ERROR: Copying /usr/local/share/java/sleuthkit-4.9.0.jar to /home/misha/Downloads/autopsy-git/autopsy/modules/ext/sleuthkit-postgresql-4.9.0.jar failed.
Please check your permissions.

Is there a way to test that fix on Autopsy 4.14?

@bcarrier Yes, I can replicate the issue with both a simple close/reopen of Autopsy itself and closing/reopening the case (with the menu Case > Close Case followed by Case > Open Case and navigating to training.aut).

Another minor oddity I've noticed is that when I run the Recent Activity module, the progress bar in the bottom-right corner never moves from 0%. I left it running for approximately 9 hours overnight, and woke up to a 0% bar. For reference, running the first few modules in the training got through to the required 15% in about 20 minutes.

markmckinnon commented 4 years ago

@mishaturnbull there is no way to test the regripper fix in 4.14 unless you want to build Autopsy from source. It will be in the 4.15 release. Also running sleuthkit 4.9.0 against Autopsy 4.14 may have issues since each version of Autopsy relies on a specific version of sleuthkit. If you build Sleuthkit then you will need to build Autopsy to avoid any issues.

mishaturnbull commented 4 years ago

Yup, that's what I tried to do. in my previous response. I cloned, built, and installed Sleuthkit per the instructions at INSTALL.TXT (which appeared to be successful) and did the same for Autopsy via Running_Linux_OSX.txt. When I tried to run it, I got the forementioned permissions error. Guessing from d5ed936, 4.15 may be coming soon?

globeone commented 4 years ago

Reproduceable

I'm able to reproduce this error as well on the training image on Ubuntu 18.04.4 LTS. I've gotten a bit further hopefully some of this analysis will help in solving the issue.

So in order to get RegRipper, the 2013! version* packaged with Autopsy in the rr-full directory, to even run at all I needed to add the path to the PERL5LIB variable.

image

Add to ~/.bashrc

I added the following to my ~/.bashrc

# Needed to run the Regripper perl modules with Autopsy export PERL5LIB="/home/$USER/programs/autopsy-4.14.0/autopsy/rr-full/"

Now I can run RegRipper by hand (outside of autopsy) on the extracted NTUSER.dat file from the trainging in: /srv/autopsycursus/Cases/case1/Temp/RecentActivity/reg/NTUSER.dat

image

CSV output of RegRipper (2013) by hand

the outputted CSV file comes back with some very interesting error messages.

ntuser.dat.csv.txt

SHA256 Sum

3ab304bc622d2b9f972b65997b2c9ca8bb690af94720678673250294f0d77c37 ntuser.dat.csv.txt

Errors

notably: Line 130 Error: Global symbol "%str" requires explicit package name (did you forget to declare "my %str"?) at /home/$USER/programs/autopsy-4.14.0/autopsy/rr-full/plugins/comdlg32.pl line 389. Compilation failed in require at rip.pl line 92.

Line 211 Error: Can't locate object method "getHive" via package "real_profilelist" (perhaps you forgot to load "real_profilelist"?) at rip.pl line 93.

Line 271 Error: Can't locate shellitems.pl in @INC (@INC contains: /home/$USER/programs/autopsy-4.14.0/autopsy/rr-full/ /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.26.1 /usr/local/share/perl/5.26.1 /usr/lib/x86_64-linux-gnu/perl5/5.26 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.26 /usr/share/perl/5.26 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /home/$USER/programs/autopsy-4.14.0/autopsy/rr-full/plugins/shellbags_test.pl line 12. Compilation failed in require at rip.pl line 92.

One food for thought:

*Throughout the forums I'm also reading a common theme of programming to a specific version. This is surprising from other security researchers as it means that the software is not easily patchable and must be hell to maintain; especially if a vulnerability in one of the point versions is found. Using RegRipper from 2013 feels inappropriate when there is a 2020 version available, especially since Windows and the Windows Registry have changed quite a bit since 2013.

https://github.com/keydet89/RegRipper2.8

AppImage (maybe for a new thread)

The other option would be to package Autopsy as an AppImage. https://appimage.org Then there are no dependency issues and it will run on any Linux platform just by double-clicking on the program. Works similar to a Mac DiskImage

globeone commented 4 years ago

Log Files

Here are the most recent Logs: autopsy.log.0.txt autopsy.log.1.txt

SHA256 Hashes

1578608833ed2ab45204b5429a292454eb09914613115d1696a06d43c58cc906 autopsy.log.0.txt a48681a49bf78831f97e27e00458e22561bddc4bd37f0e0b27efdceed20818cb autopsy.log.1.txt

jonas-koeritz commented 4 years ago

I compiled Autopsy 4.15.0 on my Arch machine and can confirm that the Recent Actvities are now working. I fixed another error in #5827.

webhat commented 4 years ago

In the file ModuleOutput/RecentActivity/reg/NTUSER.DAT-regripper-53317-full.err.txt I also get the following, which I think might be related to RegRipper issues.

Can't locate Parse/Win32Registry.pm in @INC (you may need to install the Parse::Win32Registry module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.28.1 /usr/local/share/perl/5.28.1 /usr/lib/x86_64-linux-gnu/perl5/5.28 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.28 /usr/share/perl/5.28 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /***/autopsy/autopsy-4.14.0/autopsy/rr-full/rip.pl line 28.
BEGIN failed--compilation aborted at /***/autopsy/autopsy-4.14.0/autopsy/rr-full/rip.pl line 28.
jonas-koeritz commented 4 years ago

That's a known problem and can be fixed by installing the module mentioned in the error message.

webhat commented 4 years ago

That's a known problem and can be fixed by installing the module mentioned in the error message.

I patched the script to load the perl module included with the install, rather have having 2 versions of the module installed.

    use File::Basename;
    use lib dirname(__FILE__);
mishaturnbull commented 4 years ago

Yep, I can confirm this is now working in 4.15. Sorry about the delay, finals hit me pretty hard. I haven't been able to replicate the regripper issues that were mentioned earlier -- since this seems to be resolved in 4.15, if nobody objects I can close the issue?