[New Integration] AWS Config

[cpascale43]

Description

AWS Config provides configurations and compliance statuses of AWS resources within an account. It records configuration changes and generates snapshots of the current configuration state of resources. This helps analyze AWS environments to ensure compliance and identify potential security issues.

The following integration criteria comes from the Cloud Security team, which we can expand and collaborate on as we begin this work:

• Ingest AWS Config Findings, Rules and Relationship between assets to help users enrich more context in CSPM workflows.
• Surface AWS Config mappings to various compliance benchmarks (like CIS and PCI-DSS) into Elastic, aligning with our CSPM strategy to help customers achieve compliance.
• Enumerate rules, findings and resource inventory data on the relevant CSPM Kibana pages, including benchmark rules and posture findings pages.
• Surface configuration timeline data as a new resource category to show historical compliance changes within Elastic CSPM.

Architecture

The integration should collect data exclusively through the AWS API to enable agentless ingestion in the future. The collected data will include findings (latest and historical context to support drift detection in the future), rules and relationships between assets. The relevant APIs are listed below:

Resource Relationship

AWS Config captures resource relationships (e.g., which security group is associated with an EC2 instance). This is useful in Resource Dependency Mapping, where AWS Config data creates visual maps of resource relationships. This might be relevant for graph visualization in the Findings insights workflow. API Reference for Relationship

Compliance Rules

WS Config evaluates resource configurations against predefined rules. These rules can be ingested and saved in the Elastic Rules workflow, providing users with more control over findings management within the platform. API Reference for ConfigRules

Findings

We may need to combine data from various APIs to create the posture log from AWS Config:

• ComplianceByConfigRule/ByResource
• EvaluationResult APIs
• Remediation Configuration
• ConfigRules

Metrics

The integration should provide various metrics to users which we will need to create, such as compliance scores and notification failures. These metrics include:

• aws.config.change_notifications_delivery_failed (count): Measures the number of failed attempts to deliver change notifications to an Amazon SNS topic.
• aws.config.compliance_score (gauge): Indicates the percentage of compliant rule-resource combinations in a conformance pack.
• aws.config.config_history_export_failed (count): Tracks the number of unsuccessful attempts to export configuration history to an S3 bucket.
• aws.config.config_snapshot_export_failed (count): Counts the failures in exporting configuration snapshot files to an S3 bucket.
• aws.config.configuration_items_recorded (count): Logs the number of configuration items recorded for each resource or all resource types.
• aws.config.configuration_recorder_insufficient_permissions_failure (count): Monitors the number of permission failures due to insufficient IAM role policies for the configuration recorder.

Refs

• Understanding the differences between configuration history and configuration snapshot files in AWS Config
• Delivering Configuration Snapshots to an Amazon S3 Bucket
• AWS Config API Reference

Dashboards

Dashboards provide insight into resource configurations, compliance status and changes over time. Some specific ideas are:

• Resource Inventory: Provide a comprehensive view of all AWS resources, highlighting configuration states and changes.
• Compliance Overview: Display a summary of compliant vs. non-compliant resources, including a historic trend of compliance status over time.
• Top Non-Compliant Resources: Identify and highlight resources that frequently fail compliance checks.
• Drilldown: Offer insights into specific areas like IAM configuration changes, S3 bucket policies and security group modifications.

Integration release checklist

This checklist is intended for integrations maintainers to ensure consistency when creating or updating a Package, Module or Dataset for an Integration.

All changes

• [ ] Change follows the contributing guidelines
• [ ] Supported versions of the monitoring target are documented
• [ ] Supported operating systems are documented (if applicable)
• [ ] Integration or System tests exist
• [ ] Documentation exists, useful guidelines to follow
• [ ] Fields follow ECS and naming conventions
• [ ] At least a manual test with ES / Kibana / Agent has been performed.
• [ ] Required Kibana version set to:

[terrancedejesus]

@cpascale43 - Happy to sync on this for threat detection as we are during AWS prebuilt rule dev right now.