DFIRKuiper / Hoarder

16 stars 4 forks source link

Hoarder

hoarder logo

Hoarder is a script made to collect and parse the most valuable artifacts for forensics or incident response investigations rather than imaging the whole hard drive, now it combined with Rhaegal to add detection feature

Introduction

To add easy of use and help analysts, we combined multiple tools in same repo., also the results stored in .kjson files, which can be pushed to Kuiper for visulization.

hoarder logo

Executable Releases:

You may find the latest windows binary release here

Note on 32-bit release: as of Hoarder 4.0.0, the 32-bit binary is no longer released. If you want to run hoarder in 32-bit endpoint, you can refer to the latest 32-bit release.

Usage

Collection

Hoarder parses the configuration Hoarder.yml and produces an extensive help message for ease of use.

.\hoarder.exe -h
usage: hoarder.exe [-h] [-V] [-v] [-vv] [-a] [-f IMAGE_FILE] [-pa] [-n] [-sp {1,2,3,4,5,6}] [-r] [-p] [-s] [--Events] [--Ntfs] [--Recent] [--Startup] [--SRUM] [--Firwall] [--CCM] [--WindowsIndexSearch]
                   [--Config] [--Ntuser] [--applications] [--usrclass] [--PowerShellHistory] [--RecycleBin] [--WMI] [--scheduled_task] [--BMC] [--prefetch] [--WMITraceLogs] [--BrowserHistory] [--WERFiles]
                   [--BitsAdmin] [--SystemInfo] [-g [GROUPS [GROUPS ...]]]

Hoarder 4.5.0 is a tool to collect and parse windows artifacts.

optional arguments:
  -h, --help            show this help message and exit
  -V, --version         Print Hoarder version number
  -v, --verbose         Print details of hoarder message in console
  -vv, --very_verbose   Print more details (DEBUG) of hoarder message in console
  -a, --all             Get all (Default)
  -f IMAGE_FILE, --image_file IMAGE_FILE
                        Use disk image as data source instead of the live machine disk image
  -pa, --parse_artifacts
                        Parse artifacts
  -n, --no_raw_files    Only bring parsed output. Do not bring any raw evidence files
  -sp {1,2,3,4,5,6}, --set_priority {1,2,3,4,5,6}
                        Will run Hoarder process with the selected priority. 1: BELOW_NORMAL_PRIORITY_CLASS 2: IDLE_PRIORITY_CLASS (default) 3: NORMAL_PRIORITY_CLASS 4: ABOVE_NORMAL_PRIORITY_CLASS 5:
                        HIGH_PRIORITY_CLASS 6: REALTIME_PRIORITY_CLASS
  -r, --Rhaegal         Enable detection using Rhaegal rules, only possible if parsing enabled (-sp), default not enabled
  -g [GROUPS [GROUPS ...]], --groups [GROUPS [GROUPS ...]]
                        Specify what to collect by group tag. takes a space seperated list of groups. e.g. -g execution user_activities. Available groups: ['parsing']

Plugins:
  -p, --processes       Collect information about the running processes.
  -s, --services        Collect information about the system services.

Artifacts:
  --Events              Windows event logs
  --Ntfs                $MFT file
  --Recent              Recently opened files
  --Startup             Startup info
  --SRUM                SRUM folder
  --Firwall             Firewall Logs
  --CCM                 CCM Logs
  --WindowsIndexSearch  Windows Search artifacts
  --Config              System hives
  --Ntuser              All users hives
  --applications        Amcache files
  --usrclass            UserClass.dat file for all the users
  --PowerShellHistory   PowerShell history for all the users
  --RecycleBin          RecycleBin Files
  --WMI                 WMI OBJECTS.DATA file
  --scheduled_task      Scheduled Tasks files
  --BMC                 BMC files for all the users
  --prefetch            Prefetch files
  --WMITraceLogs        WMI Trace Logs
  --BrowserHistory      BrowserHistory Data
  --WERFiles            Windows Error Reporting Files
  --BitsAdmin           Bits Admin Database (QMGR database)

Commands:
  --SystemInfo          Get system information

Examples

Let's say you want to collect all of the artifacts specified in Hoarder.yml then all you need to do is:

> .\hoarder.exe --all or > .\hoarder.exe -a or simply > .\hoarder.exe :).

After the script finishes, it will generate a zip file called <HOSTNAME>.zip containing all of the artifacts in addition to hoarder.log that contains the script debugging logs (a JSON log will also be packaged in the .zip results).

To collect all artifacts with the group tag parsing, run the following command: > .\hoarder.exe -g parsing

To run hoarder to collect Ntfs and Events artifacts, run the following command: > .\hoarder.exe --Ntfs --Events

Note: you can combine multiple groups, multiple artifacts, or even groups and artifacts, and expect hoarder to come up with a unique set of all artifacts you want to collect.

Configuration (Hoarder.yml)

Hoarder comes with the default configuration Hoarder.yml embedded in the release, if you want to use different configuration, modify the Hoarder.yml and re-combile it (check build binaries section).

  1. If you are running from the the binary executable: Hoarder.yml default configuration is embedded in it. If you place your own Hoarder.yml next to hoarder.exe it will use it instead of the default configuration.
  2. If you are running from source: you can modify Hoarder.yml or rename it and name your own configuration Hoarder.yml.

Add an Artifact to Hoarder.yml

Tip: Refer to the default configuration for reference

File and Folder Artifacts

The following is an example of file or folder collection and parsing:

  Events: 
      output: 'Events'
      path32: '\windows\system32\winevt\Logs\'
      path64: '\windows\system32\winevt\Logs\'
      groups: 'parsing'
      files: '*'
      parsers: '<|parsingdir|>MasterParser.exe -p Events -i <|path|\> -o <|output|evtx.kjson>'
      description: 'Windows event logs'
    parsers: '<|parsingdir|>MasterParser.exe -p Events -i <|path|\> -o <|output|evtx.kjson>'

It can also be a list. Example:

    parsers:
        - '<|parsingdir|>MasterParser.exe -p WMI_Persistence -i <|path|OBJECTS.DATA> -o <|output|WMI_Persistence.kjson>'
        - '<|parsingdir|>MasterParser.exe -p RUA -i <|path|OBJECTS.DATA> -o <|output|RUA.kjson>'

Parsers directives:

Parsing

Starting from hoarder 4.0.0, hoarder supports parsing collected artifacts. There are three major parts to parsing:

  1. parsers.zip: contains all of your parser binaries, scripts, and data files. Hoarder binary release comes prepackaged with MasterParser. To add your own parsers, place a parsers.zip next to hoarder.exe containing all of your parsers used. If you are running from source create your own parsers.zip and rename or delete the default parsers.zip.
  2. configuration: in Hoarder.yml, add your parser command (refer to Add an Artifact to Hoarder.yml)
  3. command-line arguments: -pa to have hoarder bring both raw and parsed artifacts. -n to have hoarder bring only parsing results. Parsing arguments work with other switches as you would expect. Meaning, If a group or a single artifact is chosen, parsing will only be applied to the chosen artifacts.
  4. Rhaegal rules: -r this option enable running the detection on parsed results, if parsing not enabled, detection will not applied.

Plugins and Commands:

Command Execution

Hoarder also support the execution of system commands. The following example shows the execution of the command "systeminfo":

  SystemInfo:
    output: 'SystemInfo'
    cmd: 'systeminfo'
    description: 'Get system information'

Running and freezing from source

Rhaegal Rules

To write a Rhaegal rules, following the schema

public hoarder_executed
{
    metadata:
      author: "Saleh Bin Muhaysin"
      creationDate: "07/11/2020"
      score: 60
      description: "Detect running hoarder on the system"
    channel: "Amcache"
    include:
      Path: "*hoarder.exe"
    exclude:
      Path: "*Desktop*"
}

Installing Dependencies

To install all Hoarder dependencies, run the following command from an elevated terminal:

pip install -r requirements.txt

compile the executable

Make sure your environment or virtual environment is setup with Python 3.8.5

To compile hoarder script, run :

pyinstaller.exe hoarder.spec

To comipe MasterParser script, run:

pyinstaller.exe MasterParser.spec

Use the tool pyinstaller to compile the executable incase of the following

Freezing Hoarder into a binary

If you want to freeze your own binary make sure you install PyInstaller 3.6. You may need to change or add to this command if your changes contain hidden imports, data files, or dll dependencies that PyInstaller needs to know about.

Pull requests

Pull requests are welcome! They will be tested then applied. We'll try to do some simple modifications if testing is not successful depending on the amount of modifications needed.

License

This project is licensed under GNU General Public License v3.0

Related Projects

Contributors:

This project built by the team (except some of the parsers).