EricZimmerman / KapeFiles

This repository serves as a place for community created Targets and Modules for use with KAPE.
MIT License
628 stars 189 forks source link

How do I configure a tkape config to search for already-collected (isolated) files (not in a windows directory structure)? #910

Closed jt0dd closed 5 months ago

jt0dd commented 5 months ago

Let's say I've collected an Amcache.hve file on its own and want to point EZTools' AmcacheParser at it. I thought I could do something like this (with --tsource C:\Users\jonat\Documents\DIGITAL_FORENSICS_DATA and Amcache being a subdirectory containing the Amcache.hve file):

Author: Eric Zimmerman
Version: 1.0
Id: 13ba1e33-4899-4843-adf1-c7e6b20d759a
RecreateDirectories: true
Targets:
    -
        Name: Amcache
        Category: ApplicationCompatibility
        Path: ./Amcache
        FileMask: Amcache.hve
    -
        Name: Amcache
        Category: ApplicationCompatibility
        Path: ./Amcache
        FileMask: Amcache.hve
    -
        Name: Amcache transaction files
        Category: ApplicationCompatibility
        Path: ./Amcache
        FileMask: Amcache.hve.LOG*
    -
        Name: Amcache transaction files
        Category: ApplicationCompatibility
        Path: ./Amcache
        FileMask: Amcache.hve.LOG*

But (running .\kape.exe --tsource C:\Users\jonat\Documents\DIGITAL_FORENSICS_DATA --tdest C:\Users\jonat\Documents\kape_out --tflush --target Amcache --mdest C:\Users\jonat\Documents\kape_mod_out --mflush --module AmcacheParser --gui) I get the output:

Using Target operations
  Flushing target destination directory C:\Users\jonat\Documents\kape_out
  Creating target destination directory C:\Users\jonat\Documents\kape_out
Found 4 targets. Expanding targets to file list...
Found 0 files

Using Module operations
  Setting --msource to C:\Users\jonat\Documents\kape_out since --msource was not provided
  Flushing module destination directory C:\Users\jonat\Documents\kape_mod_out
  Creating module destination directory C:\Users\jonat\Documents\kape_mod_out
    Found processor Executable: AmcacheParser.exe, Cmd line: -f %sourceFile% --csv %destinationDirectory% -i --mp, Export: csv, Append: False!
Discovered 1 processor to run
Executing modules with file masks...
  Skipping AmcacheParser.exe: No matching files found for Amcache.hve!
Executing remaining modules...
Executed 1 processor in 0.0138 seconds

Is there some easy way to do this?

EricZimmerman commented 5 months ago

why call a target at all? it aint gonna find it in a non standard path

.\kape.exe --msource C:\Users\jonat\Documents\DIGITAL_FORENSICS_DATA  --mdest C:\Users\jonat\Documents\kape_mod_out --mflush --module AmcacheParser

module it directly. bingo

jt0dd commented 5 months ago

Perfect. Oddly, now the resulting process execution folder is empty. But no errors:

KAPE version 1.3.0.2, Author: Eric Zimmerman, Contact: https://www.kroll.com/kape (kape@kroll.com)

KAPE directory: C:\Users\jonat\Downloads\kape\KAPE
Command line:   --msource C:\Users\jonat\Documents\DIGITAL_FORENSICS_DATA --mdest C:\Users\jonat\Documents\kape_mod_out --mflush --module AmcacheParser --gui

Using Module operations
  Flushing module destination directory C:\Users\jonat\Documents\kape_mod_out
  Creating module destination directory C:\Users\jonat\Documents\kape_mod_out
    Found processor Executable: AmcacheParser.exe, Cmd line: -f %sourceFile% --csv %destinationDirectory% -i --mp, Export: csv, Append: False!
Discovered 1 processor to run
Executing modules with file masks...
  Running AmcacheParser.exe: -f "C:\Users\jonat\Documents\DIGITAL_FORENSICS_DATA\Amcache\Amcache.hve" --csv C:\Users\jonat\Documents\kape_mod_out\ProgramExecution -i --mp
Executing remaining modules...
Executed 1 processor in 1.8772 seconds

Total execution time: 1.8906 seconds

@EricZimmerman

EricZimmerman commented 5 months ago

Use the --debug switch and see what shows up

jt0dd commented 5 months ago
KAPE directory: C:\Users\jonat\Documents\kape\KAPE
Command line:   --msource C:\Users\jonat\Documents\DIGITAL_FORENSICS_DATA --mdest C:\Users\jonat\Documents\kape_mod_out --mflush --module AmcacheParser --gui --debug

System info: 64-bit: True, User: jonat OS: Windows10 (10.0.22631)
  Validating configuration files
305 targets and 397 modules validated successfully

Using Module operations
  Flushing module destination directory C:\Users\jonat\Documents\kape_mod_out
  Creating module destination directory C:\Users\jonat\Documents\kape_mod_out
  AmcacheParser (v1.1): Determining correct processor based export type csv...
    Found processor Executable: AmcacheParser.exe, Cmd line: -f %sourceFile% --csv %destinationDirectory% -i --mp, Export: csv, Append: False!
Discovered 1 processor to run
Module name: AmcacheParser, Processor: Executable: AmcacheParser.exe, Cmd line: -f %sourceFile% --csv %destinationDirectory% -i --mp, Export: csv, Append: False , File mask: Amcache.hve , Category: ProgramExecution
Executing modules with file masks...
Getting files from C:\Users\jonat\Documents\DIGITAL_FORENSICS_DATA. Recursive: True, Mask: Amcache.hve
Found match: C:\Users\jonat\Documents\DIGITAL_FORENSICS_DATA\Amcache\Amcache.hve
Found the following files for mask Amcache.hve: C:\Users\jonat\Documents\DIGITAL_FORENSICS_DATA\Amcache\Amcache.hve
  Running AmcacheParser.exe: -f "C:\Users\jonat\Documents\DIGITAL_FORENSICS_DATA\Amcache\Amcache.hve" --csv C:\Users\jonat\Documents\kape_mod_out\ProgramExecution -i --mp
AmcacheParser version 1.5.1.0
Author: Eric Zimmerman (saericzimmerman@gmail.com)
https://github.com/EricZimmerman/AmcacheParser
Command line: -f C:\Users\jonat\Documents\DIGITAL_FORENSICS_DATA\Amcache\Amcache.hve --csv C:\Users\jonat\Documents\kape_mod_out\ProgramExecution -i --mp
Registry hive is dirty and no transaction logs were found in the same directory! LOGs should have same base name as the hive. Aborting!!
Registry hive is dirty and no transaction logs were found in the same directory! LOGs should have same base name as the hive. Aborting!!
Executing remaining modules...
Executed 1 processor in 0.6497 seconds

Total execution time: 0.6625 seconds
EricZimmerman commented 5 months ago

There's your answer.

You need to include the transaction logs or add the no log switch and everything should work just fine

jt0dd commented 5 months ago

According to GPT-4 (lol):

"The critical output message from the AmcacheParser utility is as follows:

Registry hive is dirty and no transaction logs were found in the same directory! LOGs should have the same base name as the hive. Aborting!!

This message indicates a specific problem. The Amcache.hve file, which is a registry hive, is marked as "dirty." In the context of Windows Registry hives, a "dirty" flag indicates that the hive was not properly closed by the system; in other words, it could have been in use or being written to at the time it was copied or the system did not shut down cleanly. This state necessitates special handling because simply parsing the hive as-is could lead to missing or corrupted data.

Furthermore, the message indicates that no transaction logs were found in the same directory as the Amcache.hve file. In the Windows Registry system, transaction logs (with extensions like .LOG, .LOG1, .LOG2, etc.) are used to record changes that occur within the hive."

That makes sense intuitively, because while I don't see any transaction logs (only other file is Amcache.hve.tmp) in the orig Amcache directory, I am using shadow copy to pull these hives while in use.

jt0dd commented 5 months ago

There's your answer.

You need to include the transaction logs or add the no log switch and everything should work just fine

I must be blind today, I just scrubbed website docs and -h help output without seeing anything about a no log switch. Tried --nolog and got same abort.

EricZimmerman commented 5 months ago

Run amcache parser and look at the switches.

It's probably -nologs

jt0dd commented 5 months ago

Run amcache parser and look at the switches.

It's probably -nologs

Thanks <3

Description: 'AmcacheParser: extract program execution information'
Category: ProgramExecution
Author: Eric Zimmerman
Version: 1.1
Id: 4190c518-524f-4623-8038-a014784c018c
BinaryUrl: https://f001.backblazeb2.com/file/EricZimmermanTools/AmcacheParser.zip
ExportFormat: csv
FileMask: Amcache.hve
Processors:
    -
        Executable: AmcacheParser.exe
        CommandLine: -f %sourceFile% --csv %destinationDirectory% -i --mp --nl
        ExportFormat: csv

Did the trick! Sorry to ask to be so spoonfed, I'm just on a tight time crunch trying to pitch the value of this tool to my org with a proof of concept.

EricZimmerman commented 5 months ago

No problem. Let me know if I can help you with anything else for the pilot

EricZimmerman commented 5 months ago

i guess you reached out to someone randomly at kroll. they had no idea who you were and contacted me. nowhere in your message did you state what you were working on and i helped as best i could. if you need more immediate help, simply email me at my gmail account thats plastered all over my tools.

jt0dd commented 5 months ago

i guess you reached out to someone randomly at kroll. they had no idea who you were and contacted me. nowhere in your message did you state what you were working on and i helped as best i could. if you need more immediate help, simply email me at my gmail account thats plastered all over my tools.

Yeah you did help plenty and fast. That was just during an hour where you were likely busy and I realized (not knowing if your next reply would be an hour, day, or week later) if I didn't figure this proof of concept out soon I might not make my deadline to get this tech assessed by my organization for potential use to solve a gap. I thought I might need more assistance than I ended up needing.

But you solved my problem quickly. I will reach out if I need anything else. Thanks Eric. You're awesome.

jt0dd commented 5 months ago

@EricZimmerman And the project is an automated pipeline to collect digital forensics artifacts at scale, parse them, convert outputs to JSON, and building Logstash configs to transform / ingest the useful ones to Elastic Search to be analyzed in Kibana without having to retrain analysts on the process of parsing digital forensics artifacts with various tools or even on using / configuring KAPE.

EricZimmerman commented 5 months ago

microsoft uses kape to do forensics at scale inside azure, so it should work for you =)