search

nccgroup / Scout2

Security auditing tool for AWS environments
http://nccgroup.github.io/Scout2/
GNU General Public License v2.0
1.73k stars 300 forks source link

Matching EC2 instances and IAM roles error #313

Closed czapajew closed 6 years ago

czapajew commented 6 years ago

While running Scout 2 output contain errors: CentOS Linux release 7.4.1708 (Core) Python 3.6.0

boto3 (1.9.26)
netaddr (0.7.19)
opinel (3.3.4)
python-dateutil (2.7.3)

Fetching CloudFormation config...
             regions             stacks
               15/15              20/20
Fetching CloudTrail config...
             regions             trails
               15/15                2/2
Fetching CloudWatch config...
             regions             alarms
               15/15                8/8
Fetching Direct Connect config...
             regions        connections
               15/15                0/0
Fetching EC2 config...
             regions          instances    security_groups            volumes          snapshots network_interfaces
               15/15              31/31              79/79              49/49              70/70              50/50
Fetching EFS config...
             regions       file_systems
               10/10                0/0
Fetching ElastiCache config...
             regions           clusters    security_groups
               15/15                2/2                0/0
Fetching ELB config...
             regions               elbs
               15/15                4/4
Fetching ELBV2 config...
             regions                lbs       ssl_policies
               15/15                1/1                7/7
Fetching EMR config...
             regions           clusters
               15/15                0/0
Fetching IAM config...
              groups           policies              roles              users  credential_report    password_policy
               11/11            117/117              75/75              30/30                1/1                1/1
Fetching Lambda config...
             regions          functions
               15/15              50/50
Fetching RedShift config...
             regions           clusters   parameter_groups    security_groups
               15/15                0/0                0/0                0/0
Fetching RDS config...
             regions          instances    security_groups          snapshots   parameter_groups      subnet_groups
               15/15                2/2              15/15                9/9                3/3                3/3
Fetching Route53 config...
        hosted_zones
                 3/3
Fetching Route53Domains config...
             domains
                 0/0
Fetching S3 config...
             buckets
               34/34
Fetching SES config...
             regions         identities
                 3/3                1/1
Fetching SNS config...
             regions             topics      subscriptions
               15/15                3/3                3/3
Fetching SQS config...
             regions             queues
               15/15                3/3
Fetching VPC config...
             regions       network_acls               vpcs          flow_logs            subnets  customer_gateways       vpn_gateways    vpn_connections       route_tables peering_connections
               15/15              21/21              21/21                0/0              63/63                0/0                0/0                0/0              25/25                0/0
Processing CloudTrail config...
Matching EC2 instances and IAM roles...
Traceback (most recent call last):
  File "/home/scout/scout/lib64/python3.6/site-packages/AWSScout2/rules/preprocessing.py", line 632, in new_go_to_and_do
    callback(aws_config, current_config[key][value], path, current_path, value, callback_args)
  File "/home/scout/scout/lib64/python3.6/site-packages/AWSScout2/rules/preprocessing.py", line 686, in get_lb_attack_surface
    security_group_to_attack_surface(aws_config, elb_config['external_attack_surface'], public_dns, current_path, security_groups, listeners)
  File "/home/scout/scout/lib64/python3.6/site-packages/AWSScout2/rules/preprocessing.py", line 726, in security_group_to_attack_surface
    if listener > port_min and listener < port_max and \
TypeError: '>' not supported between instances of 'int' and 'str'

Path: ['services', 'elbv2', 'regions', 'eu-west-1', 'vpcs', 'vpc-xxxxxxxx', 'lbs']
Key = lbs
Value = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Path = []
Failed to parse clusters in vpc-xxxxxxxx in eu-west-1
Traceback (most recent call last):
  File "/home/scout/scout/lib64/python3.6/site-packages/AWSScout2/rules/preprocessing.py", line 394, in match_security_groups_and_resources_callback
    sg_id = resource_sg[callback_args['sg_id_attribute_name']]
KeyError: 'SecurityGroupId'

Traceback (most recent call last):
  File "/home/scout/scout/lib64/python3.6/site-packages/AWSScout2/rules/preprocessing.py", line 632, in new_go_to_and_do
    callback(aws_config, current_config[key][value], path, current_path, value, callback_args)
  File "/home/scout/scout/lib64/python3.6/site-packages/AWSScout2/rules/preprocessing.py", line 663, in get_db_attack_surface
    security_groups = current_config['SecurityGroups']
KeyError: 'SecurityGroups'

Path: ['services', 'elasticache', 'regions', 'eu-west-1', 'vpcs', 'vpc-xxxxxxx', 'clusters']
Key = clusters
Value = wimp-workshop
Path = []
Traceback (most recent call last):
  File "/home/scout/scout/lib64/python3.6/site-packages/AWSScout2/rules/preprocessing.py", line 632, in new_go_to_and_do
    callback(aws_config, current_config[key][value], path, current_path, value, callback_args)
  File "/home/scout/scout/lib64/python3.6/site-packages/AWSScout2/rules/preprocessing.py", line 659, in get_db_attack_surface
    security_group_to_attack_surface(aws_config, service_config['external_attack_surface'], public_dns, current_path, [g['VpcSecurityGroupId'] for g in security_groups], listeners)
  File "/home/scout/scout/lib64/python3.6/site-packages/AWSScout2/rules/preprocessing.py", line 726, in security_group_to_attack_surface
    if listener > port_min and listener < port_max and \
TypeError: '>' not supported between instances of 'int' and 'str'

Path: ['services', 'rds', 'regions', 'us-west-2', 'vpcs', 'vpc-xxxxxxx', 'instances']
Key = instances
Value = wiaqua
Path = []
Loading ruleset /home/scout/scout/lib/python3.6/site-packages/AWSScout2/rules/data/rulesets/default.json
Processing CloudFormation rule[cloudformation-stack-with-role.json]: "Role passed to stack"
Processing CloudTrail rule[cloudtrail-duplicated-global-services-logging.json]: "Global service logging duplicated"
Processing CloudTrail rule[cloudtrail-no-global-services-logging.json]: "Global services logging disabled"
Processing CloudTrail rule[cloudtrail-no-log-file-validation.json]: "Log file validation disabled"
Processing CloudTrail rule[cloudtrail-no-logging.json]: "Logging disabled"
Processing CloudTrail rule[cloudtrail-not-configured.json]: "Not configured"
Processing CloudWatch rule[cloudwatch-alarm-without-actions.json]: "Alarm without action"
Processing EC2 rule[ec2-default-security-group-in-use.json]: "Default security groups in use"
Processing EC2 rule[ec2-unused-security-group.json]: "Unused security groups"
Processing EC2 rule[ec2-default-security-group-with-rules.json]: "Non-empty rulesets for default security groups"
Processing EC2 rule[ec2-ebs-volume-not-encrypted.json]: "EBS volume not encrypted"
Processing EC2 rule[ec2-security-group-opens-all-ports-to-all.json]: "All ports open to all"
Processing EC2 rule[ec2-security-group-opens-known-port-to-all.json]: "MySQL port open to all"
Processing EC2 rule[ec2-security-group-opens-known-port-to-all.json]: "DNS port open to all"
Processing EC2 rule[ec2-security-group-opens-known-port-to-all.json]: "MongoDB port open to all"
Processing EC2 rule[ec2-security-group-opens-known-port-to-all.json]: "MsSQL port open to all"
Processing EC2 rule[ec2-security-group-opens-known-port-to-all.json]: "Oracle DB port open to all"
Processing EC2 rule[ec2-security-group-opens-known-port-to-all.json]: "PostgreSQL port open to all"
Processing EC2 rule[ec2-security-group-opens-known-port-to-all.json]: "RDP port open to all"
Processing EC2 rule[ec2-security-group-opens-known-port-to-all.json]: "SSH port open to all"
Processing EC2 rule[ec2-security-group-opens-known-port-to-all.json]: "NFS port open to all"
Processing EC2 rule[ec2-security-group-opens-known-port-to-all.json]: "SMTP port open to all"
Processing EC2 rule[ec2-security-group-opens-port-to-all.json]: "TCP port open to all"
Processing EC2 rule[ec2-security-group-opens-port-to-all.json]: "UDP port open to all"
Processing EC2 rule[ec2-security-group-whitelists-aws.json]: "Security group whitelists AWS CIDRs"
Processing EC2 rule[ec2-security-group-opens-all-ports-to-self.json]: "Unrestricted network traffic within security group"
Processing EC2 rule[ec2-security-group-opens-all-ports.json]: "All ports open"
Processing EC2 rule[ec2-security-group-opens-plaintext-port.json]: "FTP port open"
Processing EC2 rule[ec2-security-group-opens-plaintext-port.json]: "Telnet port open"
Processing EC2 rule[ec2-security-group-opens-port-range.json]: "Use of port ranges"
Processing ELB rule[elb-no-access-logs.json]: "Lack of access logs"
Processing ELBV2 rule[elbv2-no-access-logs.json]: "Lack of access logs"
Processing ELBV2 rule[elbv2-no-deletion-protection.json]: "Lack of deletion protection"
Processing ELBV2 rule[elbv2-older-ssl-policy.json]: "Older SSL/TLS policy"
Processing IAM rule[iam-assume-role-lacks-external-id-and-mfa.json]: "Cross-account AssumeRole policy lacks external ID and MFA"
Processing IAM rule[iam-assume-role-policy-allows-all.json]: "AssumeRole policy allows all principals"
Processing IAM rule[iam-ec2-role-without-instances.json]: "Unused role for EC2"
Processing IAM rule[iam-group-with-inline-policies.json]: "Group with inline policies"
Processing IAM rule[iam-group-with-no-users.json]: "Group with no users"
Processing IAM rule[iam-inline-policy-allows-NotActions.json]: "Inline group policy allows NotActions"
Processing IAM rule[iam-inline-policy-for-role.json]: "Inline group policy allows iam:PassRole *"
Processing IAM rule[iam-inline-policy-for-role.json]: "Inline group policy allows sts:AssumeRole *"
Processing IAM rule[iam-inline-policy-allows-NotActions.json]: "Inline role policy allows NotActions"
Processing IAM rule[iam-inline-policy-for-role.json]: "Inline role policy allows iam:PassRole *"
Processing IAM rule[iam-inline-policy-for-role.json]: "Inline role policy allows sts:AssumeRole *"
Processing IAM rule[iam-inline-policy-allows-NotActions.json]: "Inline user policy allows NotActions"
Processing IAM rule[iam-inline-policy-for-role.json]: "Inline user policy allows iam:PassRole *"
Processing IAM rule[iam-inline-policy-for-role.json]: "Inline user policy allows sts:AssumeRole *"
Processing IAM rule[iam-managed-policy-allows-NotActions.json]: "Managed policy allows NotActions"
Processing IAM rule[iam-managed-policy-for-role.json]: "Managed policy allows iam:PassRole *"
Processing IAM rule[iam-managed-policy-for-role.json]: "Managed policy allows sts:AssumeRole *"
Processing IAM rule[iam-password-policy-minimum-length.json]: "Minimum password length too short"
Processing IAM rule[iam-password-policy-no-expiration.json]: "Password expiration disabled"
Processing IAM rule[iam-password-policy-reuse-enabled.json]: "Password reuse enabled"
Processing IAM rule[iam-role-with-inline-policies.json]: "Role with inline policies"
Processing IAM rule[iam-root-account-no-mfa.json]: "Lack of MFA (root account)"
Processing IAM rule[iam-root-account-used-recently.json]: "Root account used recently"
Processing IAM rule[iam-root-account-with-active-certs.json]: "Root account has active X.509 certs"
Processing IAM rule[iam-root-account-with-active-keys.json]: "Root account has active keys"
Processing IAM rule[iam-user-no-key-rotation.json]: "Lack of key rotation (Active)"
Processing IAM rule[iam-user-no-key-rotation.json]: "Lack of key rotation (Inactive)"
Processing IAM rule[iam-user-with-multiple-access-keys.json]: "User with multiple API keys"
Processing IAM rule[iam-user-with-policies.json]: "User with inline policies"
Processing IAM rule[iam-user-without-mfa.json]: "User without MFA"
Processing IAM rule[iam-managed-policy-no-attachments.json]: "Managed policy not attached to any entity"
Processing RDS rule[rds-instance-backup-disabled.json]: "Backup disabled"
Processing RDS rule[rds-instance-no-minor-upgrade.json]: "Auto minor version upgrade disabled"
Processing RDS rule[rds-instance-short-backup-retention-period.json]: "Short backup retention period"
Processing RDS rule[rds-instance-single-az.json]: "Single AZ RDS instance"
Processing RDS rule[rds-instance-storage-not-encrypted.json]: "Instance storage not encrypted"
Processing RDS rule[rds-security-group-allows-all.json]: "Security group allows all IP addresses"
Processing RDS rule[rds-snapshot-public.json]: "Publicly accessible snapshot"
Processing RedShift rule[redshift-cluster-database-not-encrypted.json]: "Cluster database encryption disabled"
Processing RedShift rule[redshift-cluster-no-version-upgrade.json]: "Version upgrade disabled"
Processing RedShift rule[redshift-cluster-publicly-accessible.json]: "Cluster publicly accessible"
Processing RedShift rule[redshift-parameter-group-logging-disabled.json]: "User activity logging disabled"
Processing RedShift rule[redshift-parameter-group-ssl-not-required.json]: "SSL not required"
Processing RedShift rule[redshift-security-group-whitelists-all.json]: "Security group allows all"
Processing Route53 rule[route53-domain-no-autorenew.json]: "Domain not set to autorenew"
Processing Route53 rule[route53-domain-no-transferlock.json]: "Domain transfer not locked"
Processing Route53 rule[route53-domain-transferlock-not-authorized.json]: "Domain transfer lock not supported by TLD"
Processing S3 rule[s3-bucket-no-logging.json]: "Bucket access logging disabled"
Processing S3 rule[s3-bucket-no-mfa-delete.json]: "Versioned bucket without MFA delete"
Processing S3 rule[s3-bucket-no-versioning.json]: "Bucket without versioning"
Processing S3 rule[s3-bucket-no-default-encryption.json]: "Bucket without default encryption enabled"
Processing S3 rule[s3-bucket-allowing-cleartext.json]: "Bucket allowing clear text (HTTP) communication"
Processing S3 rule[s3-bucket-world-acl.json]: "Bucket world-listable (anonymous)"
Processing S3 rule[s3-bucket-world-acl.json]: "Bucket's permissions world-readable (anonymous)"
Processing S3 rule[s3-bucket-world-acl.json]: "Bucket world-writable (anonymous)"
Processing S3 rule[s3-bucket-world-acl.json]: "Bucket's permissions world-writable (anonymous)"
Processing S3 rule[s3-bucket-world-acl.json]: "Bucket world-listable"
Processing S3 rule[s3-bucket-world-acl.json]: "Bucket's permissions world-readable"
Processing S3 rule[s3-bucket-world-acl.json]: "Bucket world-writable"
Processing S3 rule[s3-bucket-world-acl.json]: "Bucket's permissions world-writable"
Processing S3 rule[s3-bucket-world-policy-arg.json]: "Delete actions authorized to all principals"
Processing S3 rule[s3-bucket-world-policy-arg.json]: "Get actions authorized to all principals"
Processing S3 rule[s3-bucket-world-policy-arg.json]: "List actions authorized to all principals"
Processing S3 rule[s3-bucket-world-policy-arg.json]: "Put actions authorized to all principals"
Processing S3 rule[s3-bucket-world-policy-arg.json]: "Manage actions authorized to all principals"
Processing S3 rule[s3-bucket-world-policy-star.json]: "All actions authorized to all principals"
Processing SES rule[ses-identity-world-policy.json]: "SendEmail authorized to all principals"
Processing SES rule[ses-identity-world-policy.json]: "SendRawEmail authorized to all principals"
Processing SNS rule[sns-topic-world-policy.json]: "Publish authorized to all principals"
Processing SNS rule[sns-topic-world-policy.json]: "Subscribe authorized to all principals"
Processing SNS rule[sns-topic-world-policy.json]: "Receive authorized to all principals"
Processing SNS rule[sns-topic-world-policy.json]: "AddPermission authorized to all principals"
Processing SNS rule[sns-topic-world-policy.json]: "RemovePermission authorized to all principals"
Processing SNS rule[sns-topic-world-policy.json]: "SetTopicAttributes authorized to all principals"
Processing SNS rule[sns-topic-world-policy.json]: "DeleteTopic authorized to all principals"
Processing SQS rule[sqs-queue-world-policy.json]: "SendMessage authorized to all principals"
Processing SQS rule[sqs-queue-world-policy.json]: "ReceiveMessage authorized to all principals"
Processing SQS rule[sqs-queue-world-policy.json]: "PurgeQueue authorized to all principals"
Processing SQS rule[sqs-queue-world-policy.json]: "DeleteMessage authorized to all principals"
Processing SQS rule[sqs-queue-world-policy.json]: "ChangeMessageVisibility authorized to all principals"
Processing SQS rule[sqs-queue-world-policy.json]: "GetQueueAttributes authorized to all principals"
Processing SQS rule[sqs-queue-world-policy.json]: "GetQueueUrl authorized to all principals"
Processing VPC rule[vpc-custom-network-acls-allow-all.json]: "Network ACLs allow all ingress traffic (custom)"
Processing VPC rule[vpc-custom-network-acls-allow-all.json]: "Network ACLs allow all egress traffic (custom)"
Processing VPC rule[vpc-default-network-acls-allow-all.json]: "Network ACLs allow all ingress traffic (default)"
Processing VPC rule[vpc-default-network-acls-allow-all.json]: "Network ACLs allow all egress traffic (default)"
Processing VPC rule[vpc-network-acl-not-used.json]: "Unused network ACLs"
Processing VPC rule[vpc-subnet-with-bad-acls.json]: "Subnet with allow all ingress NACLs"
Processing VPC rule[vpc-subnet-with-bad-acls.json]: "Subnet with allow all egress NACLs"
Processing VPC rule[vpc-subnet-without-flow-log.json]: "Subnet without a flow log"
Processing VPC rule[vpc-subnet-without-flow-log.json]: "Subnet without a flow log"
Loading ruleset /home/scout/scout/lib/python3.6/site-packages/AWSScout2/rules/data/rulesets/filters.json
Processing IAM rule[iam-role-for-service.json]: "Role for EC2"
Processing IAM rule[iam-role-for-service.json]: "Role for Lambda"
Processing IAM rule[iam-role-for-aws-account.json]: "Role for same account"
Processing IAM rule[iam-role-for-aws-account.json]: "Role for cross account"
Processing S3 rule[s3-bucket-website-enabled.json]: "Bucket with static website enabled"
Processing EC2 rule[ec2-instance-with-open-nacls.json]: "Public instance with open NACLs"
Processing EC2 rule[ec2-security-group-with-public-cidr-grant.json]: "Security group whitelists public CIDRs"
Warning, failed to load exceptions. The file may not exist or may have an invalid format.
Searching for profiles matching ['default'] in /home/scout/.aws/credentials ... 
Searching for profiles matching ['default'] in /home/scout/.aws/config ... 
Saving data to /www/inc-awsconfig/aws_config.js
Saving config...
Saving data to /www/inc-awsconfig/exceptions.js
Saving config...
Creating /www/report.html ```
x4v13r64 commented 6 years ago

Thank you for this, are you running the latest version of Scout? I believe this error was already addressed.

czapajew commented 6 years ago

Hi! Thank You for the reply, I'm running awsscout2 v. 3.2.1

ogtony commented 6 years ago

I am also getting this bug on the latest version of Scout2, relevant traceback:

Matching EC2 instances and IAM roles...
Traceback (most recent call last):
  File "/home/REDACTED/.pyenv/versions/3.6.3/lib/python3.6/site-package
t2/rules/preprocessing.py", line 645, in new_go_to_and_do
    callback(aws_config, current_config[key][value], path, current_pat
  File "/home/REDACTED/.pyenv/versions/3.6.3/lib/python3.6/site-package
t2/rules/preprocessing.py", line 674, in get_db_attack_surface
    security_group_to_attack_surface(aws_config, service_config['exter
urrent_path, [g['VpcSecurityGroupId'] for g in security_groups], liste
  File "/home/REDACTED/.pyenv/versions/3.6.3/lib/python3.6/site-package
t2/rules/preprocessing.py", line 740, in security_group_to_attack_surf
    if listener > port_min and listener < port_max and 'cidrs' in ingr
port]:
TypeError: '>' not supported between instances of 'int' and 'str'
x4v13r64 commented 6 years ago

Thanks - this will be addressed shortly

x4v13r64 commented 6 years ago

This issue is closed in https://github.com/nccgroup/ScoutSuite and considered wontfix for Scout2.