Heyo friends!

This PR reworks quite a few things concerning the collectors engine! Hope you'll like it!!

Changes

Collectors loading

First of all, I removed COLLECTOR dict and all collectors manual imports in entry.py. Instead I replaced them with the following piece of code which dynamically loads files in the donpapi/collectors folders:

def load_collectors(root, collectors_list) -> Tuple[List, List] :
    loaded_collectors = []
    available_collectors = []
    for _, collector_name, _ in iter_modules(path=[f"{root}/collectors/"]):
        available_collectors.append(collector_name)
        if "All" in collectors_list:
            loaded_collectors.append(getattr(import_module(f"collectors.{collector_name}"), collector_name))
        else:
            if collector_name in collectors_list:
                loaded_collectors.append(getattr(import_module(f"collectors.{collector_name}"), collector_name))
    return available_collectors, loaded_collectors

To do so, I had to retrieve the root dirname of donpapi using:

root = os.path.dirname(os.path.realpath(__file__))

This function returns two dicts:

available_collectors: a list of strings used to build the argparse help menu

group_attacks.add_argument('-c','--collectors', action="store", default="All", help= ", ".join(load_collectors(root, [])[0])+"[...]

loaded_collectors: the list of loaded collectors that will be run

_, collectors = load_collectors(root, options.collectors.split(","))

Doing so makes PR'ing much easier since all we'll have to do is to add a file in donpapi/collectors/, let's say PowerShellHistory.py and respect the following format:

from typing import Any
from dploot.lib.target import Target
from dploot.lib.smb import DPLootSMBConnection
from donpapi.core import DonPAPICore
from donpapi.lib.logger import DonPAPIAdapter

class PowerShellHistory:
    user_directories = ["\\Users\\{username}\\AppData\\Roaming\\Microsoft\\Windows\\PowerShell\\PSReadLine\\"]

    def __init__(self, target: Target, conn: DPLootSMBConnection, masterkeys: list, options: Any, logger: DonPAPIAdapter, context: DonPAPICore, false_positive: list, max_filesize: int) -> None:
        self.tag = self.__class__.__name__
        self.target = target
        self.conn = conn
        self.masterkeys = masterkeys
        self.options = options
        self.logger = logger
        self.context = context
        self.found = 0
        self.false_positive = false_positive
        self.max_filesize = max_filesize

    def run(self):
        self.logger.display("Defte was here")

Note that the class name MUST be the same as the collector filename (hence PowerShellHistory.py).

Much easier :D

Collector tags

Collector tags were removed for the following:

self.tag = self.__class__.__name__

Which dynamically builds the tag value based on the class name.

We can now call self.logger.secrets the following way:

self.logger.secret(f"[{credential.winuser}] {credential.target} - {credential.username}:{credential.password}", self.tag)

Instead of the old:

self.logger.secret(f"[{credential.winuser}] {credential.target} - {credential.username}:{credential.password}", TAG)

Collectors parameters mutualization

Looking at the collectors, I realised that some of them rely on a false_positive list (there will be more because of the multiple previous PR's I did). Instead of having "magic variables", I mutualized them in the entry.py file:

# Stores the list of false positives usernames:
false_positivee = [
    ".", 
    "..", 
    "desktop.ini", 
    "Public", 
    "Default", 
    "Default User", 
    "All Users", 
    ".NET v4.5", 
    ".NET v4.5 Classic"
]
# Stores the maximum filesize 
max_filesize = 5000000

All collectors inherit these two parameters and as such it will be much more easier to add new false positive users, when found, without struggling modifying all collector files manually.

What needs to be done

As I mentionned in https://github.com/login-securite/DonPAPI/pull/80 I believe it is interesting to store files found by collectors in a separate directory as well as in donpapi/loot/TARGET/[...]/. All we need to do is to add another parameter which contains the $HOME/.donpapi/loot directory and write dump files function for all collectors which will write files into their own collector directory.

For example with the PowerShellHistory module:

Doing so makes:

Grepping easier for end users
Parsing / exposing files on the GUI easier as well (TBD)

All of that I will take care of once this PR is validated :)

See ya !!

Post Scriptum

I already adapted the previously PRed collectors, this is what it looks like for the moment :eyes:

Here is what I have done concerning the write files on disks. In donpapi/libs/utils.py I added the following function:

def dump_file_to_loot_directories(local_filepath, file_content=b"") -> None:
    os.makedirs(os.path.dirname(local_filepath), exist_ok = True)
    with open(local_filepath, "wb") as f:
        f.write(file_content)

And here is how to use it with the example of the PowerShell history collector:

import os
import ntpath
from typing import Any
from dploot.lib.target import Target
from dploot.lib.smb import DPLootSMBConnection
from donpapi.core import DonPAPICore
from donpapi.lib.logger import DonPAPIAdapter
from donpapi.lib.utils import dump_file_to_loot_directories

class PowerShellHistory:
    user_directories = ["\\Users\\{username}\\AppData\\Roaming\\Microsoft\\Windows\\PowerShell\\PSReadLine\\"]

    def __init__(self, target: Target, conn: DPLootSMBConnection, masterkeys: list, options: Any, logger: DonPAPIAdapter, context: DonPAPICore, false_positive: list, max_filesize: int) -> None:
        self.tag = self.__class__.__name__
        self.target = target
        self.conn = conn
        self.masterkeys = masterkeys
        self.options = options
        self.logger = logger
        self.context = context
        self.false_positive = false_positive
        self.max_filesize = max_filesize
        self.found = 0

    def run(self):
        self.logger.display("Gathering powershell history files")
        for user in self.context.users:
            for directory in self.user_directories:
                directory_path = directory.format(username=user)
                self.dig_files(directory_path=directory_path, recurse_level=0, recurse_max=10)
        if self.found > 0:
            self.logger.secret(f"Found {self.found} powershell history files", self.tag)

    def dig_files(self, directory_path, recurse_level=0, recurse_max=10):
        directory_list = self.conn.remote_list_dir(self.context.share, directory_path)
        if directory_list is not None:
            for item in directory_list:
                if item.get_longname() not in self.false_positive:

                    new_path = ntpath.join(directory_path, item.get_longname())
                    file_content = self.conn.readFile(self.context.share, new_path)
                    self.found += 1

                    absolute_local_filepath = os.path.join(self.context.target_output_dir, *(new_path.split('\\')))
                    dump_file_to_loot_directories(absolute_local_filepath, file_content)

                    collector_dir_local_filepath = os.path.join(self.context.global_output_dir, self.tag, new_path.replace("\\", "_"))
                    dump_file_to_loot_directories(collector_dir_local_filepath, file_content)

And to do so I only had to add the following lines in donpapi/core.py:

self.global_output_dir = os.path.join(output_dir, DPP_LOOT_DIR_NAME)
self.target_output_dir = os.path.join(output_dir, DPP_LOOT_DIR_NAME, self.host)
os.makedirs(self.target_output_dir, exist_ok=True)

Since each collector inherits the self objects, it was kinda easy!

Now files are dumped as absolute path and also in the collector sub directory:

Let me know if it suits you! :)

login-securite / DonPAPI

Multiple collectors rework #92