search

guardicore / monkey

Infection Monkey - An open-source adversary emulation platform
https://www.guardicore.com/infectionmonkey/
GNU General Public License v3.0
6.69k stars 788 forks source link

ScoutSuite integration #519

Closed ShayNehmad closed 3 years ago

ShayNehmad commented 4 years ago

We'd like to use ScoutSuite to collect and present Cloud findings in our Zero Trust report.

PDR

Problem statement

The problem currently is that we don't cover the Workloads and Orchestration pillars of Zero Trust and our report can generate more value to the users, especially enterprise ones.

The business value of this will be for the Enterprise cloud users and will be measured via download #s of this new version.

Scope

MH

NTH

Out of scope

Reference materials

https://github.com/nccgroup/ScoutSuite

Specific Components PDR

Information Gathering in the Monkeys

Functional requirements

Inputs: Collect Cloud info is configured. This will run ScoutSuite. The exact software boundaries between ScoutSuite and Infection Monkey will be discussed in the DDR including if and how to update our ScoutSuite fork. Outputs: Telemetry, how many checks were performed, how many resources were found. Logs + error logs

Quality Requirements

The setup MUST BE very clearly defined and documented so users can use this feature by adding the relevant AWS role to their EC2 machines. THIS MIGHT BE INSECURE.

Zero Trust Report

Functional requirements

Inputs: Cloud data from Monkeys via telemetries. Outputs: Zero Trust Findings Logs on telemetry parsing + errors

Quality Requirements

Performance must be unhindered with MANY findings. A 2000 findings report MUST be generated in under 7 seconds.

Checklist: Requirements

The requirements checklist contains a list of questions to ask yourself about your project’s requirements.

Use the list as a sanity check at construction time to determine how solid the ground that you’re standing on is—where you are on the requirements Richter scale. Not all of the checklist questions will apply to your project. If you’re working on an informal project, you’ll find some that you don’t even need to think about. You’ll find others that you need to think about but don’t need to answer formally. If you’re working on a large, formal project, however, you may need to consider every one.

Specific Functional Requirements

Specific Nonfunctional (Quality) Requirements

Requirements Quality

Requirements Completeness

ShayNehmad commented 4 years ago

DDR

We are mostly going to talk about boundaries.

High-level design

We will program some utility functions into ScoutSuite to make it usable as a library and not only as a standalone program with HTML output. The main goal is to create a single endpoint called run_as_library which doesn't use a file output but rather returns all of the findings in a dictionary. These changes ought to be pretty generic as we'll try to PR them back to NCC.

The Monkey agent will run this function and send the results to the Island as telemetry. Since ScoutSuite's results are already separated into RedYellowGreen, we will not need to parse and choose the severity ourselves.

We need to decide what is a test, a finding and an event in the context of ScoutSuite.

Options which have been considered but rejected

DD

Affected components

ScoutSuite

TODO: Detail changes in our fork. I'm working with NCC to understand what's the best approach there.

Seems like almost no changes will be required - maybe logger config or output file path. Nothing major for sure.

We'll need to create our own subset of their test - this is a param to the main "run" function, but might get rather complex since we're choosing a subset of a lot of tests.

Monkey agent

The decision right now is to call ScoutSuite async here: https://github.com/ShayNehmad/ScoutSuite/blob/43a931fe24ae332133a098f337584baf5a4bf2ff/ScoutSuite/__main__.py#L76

If we're on AWS, almost no params.

If we're on Azure, we'll do msi=True (Managed Service Identity)

Wait for the results, read the results file (JSON) and send telemetry. If ScoutSuite failed to execute, capture the error and send that in telemetry as well.

Island

The island should get the results from telemetry, but the processing should be rather "generic" since the results are coming already with status etc. - this is UNLIKE the previous ZT tests which were collected on Monkey side and processed on Island side.

A ScoutSuite results parser + "enricher" (which ZT test each ScoutSuite test belongs to) should exist.

Report UI

No major changes will be done to the Zero Trust report or its UI.

We will add a "which environments have been recognized" panel and we need to add an infobox explaining tests are filtered based on this.

Testing

🤕 Assumption - we'll only be as good as ScoutSuite. We can open PRs to ScoutSuite to fix issues we're unhappy with.

Sanity

Set up a situation where one check is red, one is yellow, one is green, one is grey.

Edge cases

All parsing code should be edge-cased via Unit Tests and NOT manual cloud testing; this will be much more efficient.

Risks + opportunities

RISK: Cloud setups will take a lot of time to get up and running and to debug and test on. RISK: Complex parsing/translation layer from SS to IM ZT RISH: High maintenance cost of pulling back changes from ncc/scoutsuite into shay/scoutsuite OPPO: Improve report performance OPPO: Improve report loading screen with separate indicators for all the different endpoints OPPO: Improve the system_info module with a plugin architecture

Tasks

Dev

ShayNehmad commented 4 years ago

DDR 1 with @danielguardicore :

ShayNehmad commented 4 years ago

WIP

ScoutSuite Exploratory Surgery for DDR

Setup

I set up the following machines, and I'm debugging Directly from PyCharm using remote SSH shell:

Conclusions

Installations

There are quite a lot of extra dependencies this bestows on the Monkey (mostly cloud stuff)).

First execution - AWS

Went with AWS. Ran using Python 3.7 - python3.7 scout.py aws. This means we need to know ahead of time which provider we are planning to scan and pass the parameter accordingly.

Started debugging.

The main mechanism seems to be an asyncio event loop which uses a ThreadPool - they already thought about MultiThreading. Need to make sure I can run this async myself. I didn't debug the internals by accident, so let's start with analyzing the results (and the results FILE) for now and approach the internals + interesting parameters later.

Results analysis

Interesting categories

The "baseline" will be AWS (Azure equivalent will be provided later).

For sure

We'll have the following "categories", meaning tests, per cloud provider:

Maybe list

Might fits in Storage category

Might fit in general Orchastration category...

Nope

Second execution - Azure

scout.py azure: error: one of the arguments -c/--cli -m/--msi -s/--service-principal --file-auth --user-account is required.

To fix:

scout.py azure --msi uses the Managed Service Account (IAM equivalent).

Results are MUCH less extensive, seems like ScoutSuite care much less about Azure.

VakarisZ commented 4 years ago

Implementation Plan and other details

SS findings -> ZT findings translation

  1. Island receives SS findings
  2. Island goes through each SS finding, that's relevant to ZT findings, and creates new ZT finding with SS finding in details or adds SS finding to the details of an already present ZT finding. For that we also need to define which SS findings map to which ZT findings (many to one). For example ZT finding "Scoutsuite found overly permissive firewall rules" would map to SS findings: "Security Group Opens All Ports to All", "Security Group Opens RDP Port to All", "Security Group Opens SSH Port to All" etc.
  3. Island goes through each relevant service and saves it's resources in mongodb.

UI

  1. UI receives ZT findings and cloud services info.
  2. Based on ZT finding type it either renders events or resources button. image

  3. Resources button opens up a modal window with dropdowns of each SS finding. Dropdowns contain resources, relevant to SS finding. Below screenshot shows how resources would look(without dropdown) image

    Task list

Setup

Infrastructure

SS finding to ZT finding translation

SS finding display

Total estimate: 13 - 16 sp.

VakarisZ commented 4 years ago

Common terms:

Monkey finding

Monkey finding is zero trust finding created by monkey. It contains usual ZT finding structure, it's details contain events.

Monkey finding data structure corresponds to:

{
    "_id" : ObjectId("5f576dd1c62c74f8cde19244"),
    "_cls" : "Finding",
    "type": "monkey",
    "test" : "endpoint_security_exists",
    "status" : "Failed",
    "details" : {
        "_cls" : "MonkeyFindingDetails",
        "_ref" : {
            "$ref" : "monkey_finding_details",
            "$id" : ObjectId("5f576dd1c62c74f8cde19243")
        }
    }
}

Monkey finding details correspond to:

{
    "_id" : ObjectId("5f576dd1c62c74f8cde19243"),
    "events" : [ 
        {
            "timestamp" : ISODate("2020-09-08T14:41:05.791Z"),
            "title" : "Process list",
            "message" : "Monkey on ip-10-0-0-167 scanned the process list",
            "event_type" : "monkey_local"
        }, 
        {
            "timestamp" : ISODate("2020-09-08T15:05:06.774Z"),
            "title" : "Process list",
            "message" : "Monkey on ip-10-0-0-167 scanned the process list",
            "event_type" : "monkey_local"
        }
    ]
}

Scoutsuite finding

Scoutsuite finding is zero trust finding created using scoutsuite report data. It contains usual ZT finding structure, it's details contain scoutsuite rules.

Scoutsuite finding data structure corresponds to:

{
    "_id" : ObjectId("5f576dd1c62c74f8cde19244"),
    "_cls" : "Finding",
    "type": "scoutsuite",
    "test" : "scoutsuite_permissive_firewall_rules",
    "status" : "Failed",
    "details" : {
        "_cls" : "ScoutsuiteFindingDetails",
        "_ref" : {
            "$ref" : "scoutsuite_finding_details",
            "$id" : ObjectId("5f576dd1c62c74f8cde19243")
        }
    }
}

Scoutsuite finding details data structure corresponds to::

{
    "_id" : ObjectId("5f576dd1c62c74f8cde19244"),
    "_cls" : "ScoutsuiteFindingDetails",
    "scoutsuite_rules" : [
       {'description': 'Security Group Opens All Ports to All', 
        'path': 'ec2.regions.id.vpcs.id.security_groups.id.rules.id.protocols.id.ports.id.cidrs.id.CIDR', 
        'level': 'danger', 
        'display_path': 'ec2.regions.id.vpcs.id.security_groups.id', 
        'items': ['ec2.regions.eu-central-1.vpcs.vpc-00015526b6695f9aa.security_groups.sg-019eb67135ec81e65.rules.ingress.protocols.ALL.ports.1-65535.cidrs.0.CIDR'], 
        'dashboard_name': 'Rules', 
        'checked_items': 172, 
        'flagged_items': 1, 
        'service': 'EC2', 
        'rationale': 'It was detected that all ports in the security group are open, and any source IP address could send traffic to these ports, which creates a wider attack surface for resources assigned to it. Open ports should be reduced to the minimum needed to correctly operate and,  when possible, source address restrictions should be implemented.', 
        'remediation': None, 
        'compliance': None, 
        'references': None
     },
      {'description': 'Get Actions Authorized to All Principals', 
       'path': 's3.buckets.id.policy.Statement.id',
       'level': 'danger', 
       'display_path': 's3.buckets.id',
       'items': ['s3.buckets.b5db8203e27b115e4c6a99199da1e7081077ce92.policy.Statement.0'], 
       'dashboard_name': 'Buckets',
       'checked_items': 3, 
       'flagged_items': 1, 
       'service': 'S3',
       'rationale': "Allowing IAM actions to all principals is contrary to the principle of least privilege and presents and opportunity for abuse. This policy should be reviewed to ensure it is secure and in line with the resource's intended use.",
       'remediation': None, 
       'compliance': None, 
       'references': None
    }
    ]
}

Resources

Resources represent items on cloud. Resources can also contain other resources, for example regions have vpc's which have instances and so on.

Obviously, data between different resource types varies greatly.

Resources are part of data ScoutSuite generates, and will be stored on the database.

Example data of ec-2 instance: image

Example data of s3 bucket: image

ScoutSuite data -> Scoutsuite ZT finding.

Data analysis:

Scoutsuite findings are ordered by cloud service. Each service contains some form of resources (buckets, vpc's, instances, security groups etc.) and findings and/or filters (which we call scoutsuite rules because their contents follow rule data structure. image

In the ec2 service, relevant resources are regions. We should make sure to only store relevant services, and only send resources to UI once, not with every finding. image

findings/filters (same thing, we use the term rule for them) contain a variety of fields, but they are consistent throughout different findings. description, path, level, display_path, dashboard_name and rationale and some other fields seem to be present in all findings, whereas remediation, compliance and references might be 'None'. image

Items contain a path to relevant resources in scoutsuite data. This is how we know which field/resource is vulnerable/misconfigured. image

Process overview

High level overview of ZT workflow, involving ScoutSuite data parsing

  1. Scoutsuite API: if monkey is ran on island, it calls scoutsuite API to gather scoutsuite data.
  2. Scoutsuite data parsing on backend: monkey island receives scoutsuite data.
    • Island creates Scoutsuite ZT findings: Island picks out scoutsuite rules relevant to the finding out of scoutsuite report data and based on them determines the status of scoutsuite ZT finding created. Then, scoutsuite rules are aggregated into scoutsuite finding details collection, which is linked to Scoutsuite ZT finding.
    • The whole scoutsuite report data is saved in bulk on the database. We need it, because it contains resources we'll want to show in the UI.
  3. UI requests ZT report: Island returns all ZT findings: monkey ZT findings and scoutsuite ZT findings. Additionally, scoutsuite report data is also sent to UI.
  4. UI displays ZT report: Scoutsuite ZT findings are displayed as shown in the mockups above. In each scoutsuite rule only affected resources are displayed:
    • According to path, display_paht and items values of a scoutsuite rule, corresponding resource data is pulled from scoutsuite report data. We display only basic resource fields.
    • (Optional) resources are displayed similarly to ScoutSuite report, by using their handlebars templates (same as in UI mockup)