aws-samples / siem-on-amazon-opensearch-service

A solution for collecting, correlating and visualizing multiple types of logs to help investigate security incidents.
MIT No Attribution
567 stars 184 forks source link

es_loader Error: sf_securityhub.py UnboundLocalError instanceid #395

Closed dpiddock closed 1 year ago

dpiddock commented 1 year ago

Hi.

I've spotted repeated errors in the es_loader lambda logs:

Traceback (most recent call last):
  File "/var/task/siem/__init__.py", line 840, in transform_by_script
    self.__logdata_dict = self.sf_module.transform(
  File "/var/task/siem/sf_securityhub.py", line 227, in transform
    resource_dict = get_values_from_asff_resources(logdata['Resources'])
  File "/var/task/siem/sf_securityhub.py", line 92, in get_values_from_asff_resources
    resource_dict['cloud'] = {'instance': {'id': instanceid}}
UnboundLocalError: local variable 'instanceid' referenced before assignment

Caused by this line: sf_securityhub.py#L92

The event relates to an EBS volume that is not attached to an instance.

Sample event:

{
  "version": "0",
  "id": "11111111-2222-3333-4444-555555555555",
  "detail-type": "Security Hub Findings - Imported",
  "source": "aws.securityhub",
  "account": "111111111111",
  "time": "2023-08-30T09:22:04Z",
  "region": "us-west-1",
  "resources": [
    "arn:aws:securityhub:us-west-1::product/aws/securityhub/arn:aws:securityhub:us-west-1:111111111111:subscription/aws-foundational-security-best-practices/v/1.0.0/EC2.3/finding/11111111-2222-3333-4444-555555555555"
  ],
  "detail": {
    "findings": [
      {
        "ProductArn": "arn:aws:securityhub:us-west-1::product/aws/securityhub",
        "Types": [
          "Software and Configuration Checks/Industry and Regulatory Standards/AWS-Foundational-Security-Best-Practices"
        ],
        "Description": "This AWS control checks whether the EBS volumes that are in an attached state are encrypted.",
        "Compliance": {
          "Status": "NOT_AVAILABLE",
          "StatusReasons": [
            {
              "Description": "This finding has a compliance status of NOT AVAILABLE because AWS Config sent Security Hub a finding with a compliance state of Not Applicable. The potential reasons for a Not Applicable finding from Config are that (1) a resource has been moved out of scope of the Config rule; (2) the Config rule has been deleted; (3) the resource has been deleted; or (4) the logic of the Config rule itself includes scenarios where Not Applicable is returned. The specific reason why Not Applicable is returned is not available in the Config rule evaluation.",
              "ReasonCode": "CONFIG_RETURNS_NOT_APPLICABLE"
            },
            {
              "Description": "The finding is in an ARCHIVED state because the finding was generated by a security check and it hasn't been updated in 3 days. This could be due to the resource being deleted or the control being disabled.",
              "ReasonCode": "NOT_UPDATED_WITHIN_THREE_DAYS"
            }
          ],
          "SecurityControlId": "EC2.3",
          "AssociatedStandards": [
            {
              "StandardsId": "standards/aws-foundational-security-best-practices/v/1.0.0"
            }
          ]
        },
        "ProductName": "Security Hub",
        "FirstObservedAt": "2023-08-27T03:04:22.577Z",
        "CreatedAt": "2023-08-27T03:04:22.577Z",
        "LastObservedAt": "2023-08-30T09:22:01.252Z",
        "CompanyName": "AWS",
        "FindingProviderFields": {
          "Types": [
            "Software and Configuration Checks/Industry and Regulatory Standards/AWS-Foundational-Security-Best-Practices"
          ],
          "Severity": {
            "Normalized": 40,
            "Label": "INFORMATIONAL",
            "Product": 40,
            "Original": "INFORMATIONAL"
          }
        },
        "ProductFields": {
          "StandardsArn": "arn:aws:securityhub:::standards/aws-foundational-security-best-practices/v/1.0.0",
          "StandardsSubscriptionArn": "arn:aws:securityhub:us-west-1:111111111111:subscription/aws-foundational-security-best-practices/v/1.0.0",
          "ControlId": "EC2.3",
          "RecommendationUrl": "https://docs.aws.amazon.com/console/securityhub/EC2.3/remediation",
          "RelatedAWSResources:0/name": "securityhub-encrypted-volumes-4cf395f0",
          "RelatedAWSResources:0/type": "AWS::Config::ConfigRule",
          "StandardsControlArn": "arn:aws:securityhub:us-west-1:111111111111:control/aws-foundational-security-best-practices/v/1.0.0/EC2.3",
          "aws/securityhub/ProductName": "Security Hub",
          "aws/securityhub/CompanyName": "AWS",
          "aws/securityhub/annotation": "This finding has a compliance status of NOT AVAILABLE because AWS Config sent Security Hub a finding with a compliance state of Not Applicable. The potential reasons for a Not Applicable finding from Config are that (1) a resource has been moved out of scope of the Config rule; (2) the Config rule has been deleted; (3) the resource has been deleted; or (4) the logic of the Config rule itself includes scenarios where Not Applicable is returned. The specific reason why Not Applicable is returned is not available in the Config rule evaluation.",
          "Resources:0/Id": "arn:aws:ec2:us-west-1:111111111111:volume/vol-aaaaaaaaaaaaaaaaa",
          "ArchivalReasons:0/ReasonCode": "NOT_UPDATED_WITHIN_THREE_DAYS",
          "ArchivalReasons:0/Description": "The finding is in an ARCHIVED state because the finding was generated by a security check and it hasn't been updated in 3 days. This could be due to the resource being deleted or the control being disabled.",
          "aws/securityhub/FindingId": "arn:aws:securityhub:us-west-1::product/aws/securityhub/arn:aws:securityhub:us-west-1:111111111111:subscription/aws-foundational-security-best-practices/v/1.0.0/EC2.3/finding/11111111-2222-3333-4444-555555555555"
        },
        "Remediation": {
          "Recommendation": {
            "Text": "For information on how to correct this issue, consult the AWS Security Hub controls documentation.",
            "Url": "https://docs.aws.amazon.com/console/securityhub/EC2.3/remediation"
          }
        },
        "SchemaVersion": "2018-10-08",
        "GeneratorId": "aws-foundational-security-best-practices/v/1.0.0/EC2.3",
        "RecordState": "ARCHIVED",
        "Title": "EC2.3 Attached EBS volumes should be encrypted at-rest",
        "Workflow": {
          "Status": "NEW"
        },
        "Severity": {
          "Normalized": 40,
          "Label": "INFORMATIONAL",
          "Product": 40,
          "Original": "INFORMATIONAL"
        },
        "UpdatedAt": "2023-08-30T09:22:01.252Z",
        "WorkflowState": "NEW",
        "AwsAccountId": "111111111111",
        "Region": "us-west-1",
        "Id": "arn:aws:securityhub:us-west-1:111111111111:subscription/aws-foundational-security-best-practices/v/1.0.0/EC2.3/finding/11111111-2222-3333-4444-555555555555",
        "Resources": [
          {
            "Partition": "aws",
            "Type": "AwsEc2Volume",
            "Region": "us-west-1",
            "Id": "arn:aws:ec2:us-west-1:111111111111:volume/vol-aaaaaaaaaaaaaaaaa"
          }
        ],
        "ProcessedAt": "2023-08-30T09:22:03.455Z"
      }
    ]
  }
}

Thank you

nakajiak commented 1 year ago

Fixed. Thanks for the feedback. It was very helpful because the sample log was detailed and specific