cnspec does not install all required plugins in variant policies - bundle compile failure

[scottford-io]

Describe the bug Executing a cnspec policies that utilizes variants requires that ALL of plugins are installed in order for cnspec to properly parse the policy. If you install cnspec on a new system that has no plugins and try to execute a scan with a policy that has more than one target such as terraform-hcl and aws, cnspec will only automatically download the plugin that is called during a scan.

For example if a policy has variants as follows:

    variants:
      - uid: aws-security-ec2-imdsv2-check-api
      - uid: aws-security-ec2-imdsv2-check-single
      - uid: aws-security-ec2-imdsv2-check-terraform-hcl
      - uid: aws-security-ec2-imdsv2-check-terraform-plan

If you execute a scan against terraform-hcl as follows:

cnspec scan terraform </path/to/terraform> -f my-variant-policy.mql.yaml

cnspec will only download the terraform plugin and so it does not understand any of the checks written for aws which causes the scan to fail...

cnspec scan terraform ../../luna-gitlab/aws-provisioning -f policies/aws-security.mql.yaml
→ installing provider 'terraform' version=9.0.11
→ successfully installed terraform provider path=/Users/scottford/.config/mondoo/providers/terraform version=9.0.11
→ no Mondoo configuration file provided, using defaults
! No credentials provided. Switching to --incognito mode.
! refresh checksum on filter of query , which should have been pre-compiled filter="asset.platform == \"aws-security-group\" && aws.ec2.securitygroup.name == \"default\"" mql="aws.ec2.securitygroup.ipPermissions.all(ipRanges.length == 0 && ipv6Ranges.length == 0 && fromPort == 0 && toPort == 0)\naws.ec2.securitygroup.ipPermissionsEgress.all(ipRanges.length == 0 && ipv6Ranges.length == 0 && fromPort == 0 && toPort == 0)\n"
! refresh checksum on filter of query , which should have been pre-compiled filter="asset.platform == \"aws-security-group\" && aws.ec2.securitygroup.name == \"default\"" mql="aws.ec2.securitygroup.ipPermissions.all(ipRanges.length == 0 && ipv6Ranges.length == 0 && fromPort == 0 && toPort == 0)\naws.ec2.securitygroup.ipPermissionsEgress.all(ipRanges.length == 0 && ipv6Ranges.length == 0 && fromPort == 0 && toPort == 0)\n"
FTL failed to resolve policies error="failed to compile bundle: failed to compile //local.cnspec.io/run/local-execution/queries/aws-security-ec2-imdsv2-check-api: failed to compile query 'aws.ec2.instances.all( httpTokens == \"required\" )': cannot find resource for identifier 'aws'\nfailed to compile //local.cnspec.io/run/local-execution/queries/aws-security-ec2-imdsv2-check-single: failed to compile query 'aws.ec2.instance.httpTokens == \"required\"\n': failed to compile: cannot find resource for identifier 'aws'\nfailed to compile //local.cnspec.io/run/local-execution/queries/aws-security-ec2-default-security-group-no-use-api: failed to compile query 'aws.ec2.instances.all( securityGroups.none( name == \"default\" ) )\n': cannot find resource for identifier 'aws'\nfailed to compile //local.cnspec.io/run/local-execution/queries/aws-security-ec2-default-security-group-no-use-single: failed to compile query 'aws.ec2.instance.securityGroups.none( name == \"default\" )\n': cannot find resource for identifier 'aws'\nfailed to compile //local.cnspec.io/run/local-execution/queries/aws-security-restricted-ssh-api: failed to compile query 'aws.ec2.securityGroups.all( ipPermissions.none( toPort == props.sshPort ))': cannot find resource for identifier 'aws'\nfailed to compile //local.cnspec.io/run/local-execution/queries/aws-security-restricted-ssh-single: failed to compile query 'aws.ec2.securitygroup.ipPermissions.all( toPort == props.sshPort )\n': cannot find resource for identifier 'aws'\nfailed to compile //local.cnspec.io/run/local-execution/queries/aws-security-vpc-lock-default-security-group: cannot refresh checksum for query, its filters were not compiled\nfailed to compile //local.cnspec.io/run/local-execution/queries/aws-security-vpc-lock-default-security-group-api: failed to compile query 'aws.ec2.securityGroups.where(name == \"default\").all(ipPermissions.all(ipRanges.length == 0 && ipv6Ranges.length == 0 && fromPort == 0 && toPort == 0))\naws.ec2.securityGroups.where(name == \"default\").all( ipPermissionsEgress.all(ipRanges.length == 0 && ipv6Ranges.length == 0 && fromPort == 0 && toPort == 0))\n': cannot find resource for identifier 'aws'\nfailed to compile //local.cnspec.io/run/local-execution/queries/aws-security-vpc-lock-default-security-group-single: failed to compile query 'aws.ec2.securitygroup.ipPermissions.all(ipRanges.length == 0 && ipv6Ranges.length == 0 && fromPort == 0 && toPort == 0)\naws.ec2.securitygroup.ipPermissionsEgress.all(ipRanges.length == 0 && ipv6Ranges.length == 0 && fromPort == 0 && toPort == 0)\n': cannot find resource for identifier 'aws'\nfailed to compile //local.cnspec.io/run/local-execution/queries/aws-security-vpc-security-group-no-port-ranges-api: failed to compile query 'aws.ec2.securityGroups.all( ipPermissions.all( toPort == fromPort ))\n': cannot find resource for identifier 'aws'\nfailed to compile //local.cnspec.io/run/local-execution/queries/aws-security-vpc-security-group-no-port-ranges-single: failed to compile query 'aws.ec2.securitygroup.ipPermissions.all( toPort == fromPort )\n': cannot find resource for identifier 'aws'\nfailed to compile //local.cnspec.io/run/local-execution/queries/aws-security-ec2-encryption-api: failed to compile query 'aws.ec2.volumes.where( state == \"in-use\" ).all( encrypted == true )\n': cannot find resource for identifier 'aws'\nfailed to compile //local.cnspec.io/run/local-execution/queries/aws-security-vpc-security-group-description-api: failed to compile query 'aws.ec2.securityGroups.all( description != null || description != \"\" )\n': cannot find resource for identifier 'aws'\nfailed to compile //local.cnspec.io/run/local-execution/queries/aws-security-ec2-imdsv2-check-api: failed to compile query 'aws.ec2.instances.all( httpTokens == \"required\" )': cannot find resource for identifier 'aws'\nfailed to compile //local.cnspec.io/run/local-execution/queries/aws-security-ec2-imdsv2-check-single: failed to compile query 'aws.ec2.instance.httpTokens == \"required\"\n': failed to compile: cannot find resource for identifier 'aws'\nfailed to compile //local.cnspec.io/run/local-execution/queries/aws-security-ec2-default-security-group-no-use-api: failed to compile query 'aws.ec2.instances.all( securityGroups.none( name == \"default\" ) )\n': cannot find resource for identifier 'aws'\nfailed to compile //local.cnspec.io/run/local-execution/queries/aws-security-ec2-default-security-group-no-use-single: failed to compile query 'aws.ec2.instance.securityGroups.none( name == \"default\" )\n': cannot find resource for identifier 'aws'\nfailed to compile //local.cnspec.io/run/local-execution/queries/aws-security-restricted-ssh-api: failed to compile query 'aws.ec2.securityGroups.all( ipPermissions.none( toPort == props.sshPort ))': cannot find resource for identifier 'aws'\nfailed to compile //local.cnspec.io/run/local-execution/queries/aws-security-restricted-ssh-single: failed to compile query 'aws.ec2.securitygroup.ipPermissions.all( toPort == props.sshPort )\n': cannot find resource for identifier 'aws'\nfailed to compile //local.cnspec.io/run/local-execution/queries/aws-security-vpc-lock-default-security-group: cannot refresh checksum for query, its filters were not compiled\nfailed to compile //local.cnspec.io/run/local-execution/queries/aws-security-vpc-lock-default-security-group-api: failed to compile query 'aws.ec2.securityGroups.where(name == \"default\").all(ipPermissions.all(ipRanges.length == 0 && ipv6Ranges.length == 0 && fromPort == 0 && toPort == 0))\naws.ec2.securityGroups.where(name == \"default\").all( ipPermissionsEgress.all(ipRanges.length == 0 && ipv6Ranges.length == 0 && fromPort == 0 && toPort == 0))\n': cannot find resource for identifier 'aws'\nfailed to compile //local.cnspec.io/run/local-execution/queries/aws-security-vpc-lock-default-security-group-single: failed to compile query 'aws.ec2.securitygroup.ipPermissions.all(ipRanges.length == 0 && ipv6Ranges.length == 0 && fromPort == 0 && toPort == 0)\naws.ec2.securitygroup.ipPermissionsEgress.all(ipRanges.length == 0 && ipv6Ranges.length == 0 && fromPort == 0 && toPort == 0)\n': cannot find resource for identifier 'aws'\nfailed to compile //local.cnspec.io/run/local-execution/queries/aws-security-vpc-security-group-no-port-ranges-api: failed to compile query 'aws.ec2.securityGroups.all( ipPermissions.all( toPort == fromPort ))\n': cannot find resource for identifier 'aws'\nfailed to compile //local.cnspec.io/run/local-execution/queries/aws-security-vpc-security-group-no-port-ranges-single: failed to compile query 'aws.ec2.securitygroup.ipPermissions.all( toPort == fromPort )\n': cannot find resource for identifier 'aws'\nfailed to compile //local.cnspec.io/run/local-execution/queries/aws-security-ec2-encryption-api: failed to compile query 'aws.ec2.volumes.where( state == \"in-use\" ).all( encrypted == true )\n': cannot find resource for identifier 'aws'\nfailed to compile //local.cnspec.io/run/local-execution/queries/aws-security-vpc-security-group-description-api: failed to compile query 'aws.ec2.securityGroups.all( description != null || description != \"\" )\n': cannot find resource for identifier 'aws'\n"

To Reproduce

Example Variant policy

policies:
    - uid: aws-security
      name: mondoo AWS Security
      version: 1.0.0
      scoring_system: 2
      authors:
      - name: mondoo, Inc.
        mondoo.com/platform: aws,cloud
        mondoo.com/category: security
      groups:
        - title: AWS EC2 Security
          checks:
            - uid: aws-security-ec2-imdsv2-check
queries:
  - uid: aws-security-ec2-imdsv2-check
    title: 'EC2 instances should use Instance Metadata Service Version 2 (IMDSv2) when IMDS is enabled'
    impact: 80
    docs:
      desc: |
        Version 2 of the IMDS adds new protections for the following types of vulnerabilities. These vulnerabilities could be used to try to access the IMDS.

          - Open website application firewalls
          - Open reverse proxies
          - Server-side request forgery (SSRF) vulnerabilities
          - Open Layer 3 firewalls and network address translation (NAT)

        Security Hub recommends that you configure your EC2 instances with IMDSv2.
    variants:
      - uid: aws-security-ec2-imdsv2-check-api
      - uid: aws-security-ec2-imdsv2-check-single
      - uid: aws-security-ec2-imdsv2-check-terraform-hcl
      - uid: aws-security-ec2-imdsv2-check-terraform-plan
  - uid: aws-security-ec2-imdsv2-check-api
    filters: asset.platform == "aws"
    mql: aws.ec2.instances.all( httpTokens == "required" )
    docs:
      remediation:
        - id: console
          desc: |
            To configure EC2 instances with IMDSv2, see [Recommended path to requiring IMDSv2](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-metadata-transition-to-version-2.html#recommended-path-for-requiring-imdsv2) in the Amazon EC2 User Guide for Linux Instances.
  - uid: aws-security-ec2-imdsv2-check-single
    filters: asset.platform == "aws-instance"
    mql: |
      aws.ec2.instance.httpTokens == "required"
  - uid: aws-security-ec2-imdsv2-check-terraform-hcl
    filters: asset.platform == "terraform-hcl" && terraform.resources.any( nameLabel == "aws_instance" )
    mql: terraform.resources.where( nameLabel == "aws_instance" ).all( blocks.one( type == "metadata_options" ) && blocks.where( type == "metadata_options" ).all( arguments['http_tokens'] == "required" || arguments['http_endpoint'] == "disabled" ))
    docs:
      remediation:
        - id: terraform-hcl
          desc: |
            #### Require the IMDSv2 on EC2 instances

            The following example requires session tokens when access the Instance Metadata Service, also known as IMDSv2.

        resource "aws_instance" "example" {
          ...

          metadata_options {
            http_tokens = "required"
          }
          ...
        }
        ```

        #### Disable the IMDS service on EC2 instances

        The following example disables the Instance Metadata Service.

        ```
        resource "aws_instance" "example" {
          ...

          metadata_options {
            http_endpoint = "disabled"
          }
          ...
        }
        ```

• uid: aws-security-ec2-imdsv2-check-terraform-plan filters: asset.platform == "terraform-plan" && terraform.plan.resourceChanges.any( type == "aws_instance" ) mql: | terraform.plan.resourceChanges.where( type == "aws_instance" ).all( change.after['metadata_options'][0]['http_tokens'] == "required" || change.after['metadata_options'][0]['http_endpoint'] == 'disabled' )

Example Terraform

resource "random_id" "random" {
  byte_length = 4
}

////////////////////////////////
// VPC Configuration

module "vpc" {
  source = "terraform-aws-modules/vpc/aws"

  name = "mainVPC-${random_id.random.id}"
  cidr = var.vpc_cidr

  azs                = ["${var.region}a", "${var.region}b", "${var.region}c"]
  private_subnets    = var.vpc_private_subnets
  public_subnets     = var.vpc_public_subnets
  enable_nat_gateway = var.vpc_enable_nat_gateway
}

////////////////////////////////
// Default Security Group

resource "aws_default_security_group" "default" {
  vpc_id = module.vpc.vpc_id

  ingress {
    protocol  = "-1"
    self      = true
    from_port = 0
    to_port   = 0
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

////////////////////////////////
// Linux Security Groups

module "allow_unrestricted_ssh" {
  source = "terraform-aws-modules/security-group/aws"

  name        = "allow-ssh-unrestricted"
  description = "Allow unrestricted access to SSH"
  vpc_id      = module.vpc.vpc_id

  ingress_with_cidr_blocks = [
    {
      from_port   = 22
      to_port     = 22
      protocol    = "-1"
      description = "Allow unrestricted SSH access"
      cidr_blocks = "${var.myIp}"
    }
  ]

  egress_with_cidr_blocks = [
    {
      from_port   = 0
      to_port     = 0
      protocol    = "-1"
      description = "User-service ports (ipv4)"
      cidr_blocks = "0.0.0.0/0"
    },
  ]
}

resource "aws_security_group" "allow_3306" {
  name = "allow_3306"
  description = "Allow ingress 3306"
  vpc_id = module.vpc.vpc_id  
}

resource "aws_security_group_rule" "allow_3306" {
  type              = "ingress"
  from_port         = 3306
  to_port           = 3306
  protocol          = "tcp"
  cidr_blocks       = [module.vpc.private_subnets_cidr_blocks[0], "0.0.0.0/0"]
  security_group_id = aws_security_group.allow_3306.id
}

resource "aws_security_group" "allow_5432" {
  name = "allow_5432"
  description = "Allow ingress 5432"
  vpc_id = module.vpc.vpc_id  
}

resource "aws_vpc_security_group_ingress_rule" "allow_5432" {
  security_group_id = aws_security_group.allow_5432.id

  cidr_ipv4   = module.vpc.private_subnets_cidr_blocks[0]
  from_port   = 5432
  ip_protocol = "tcp"
  to_port     = 5432
}

resource "aws_security_group" "allow_tls" {
  name        = "allow_tls"
  description = "Allow unrestricted TLS inbound traffic"
  vpc_id      = module.vpc.vpc_id

  ingress {
    description = "TLS from VPC"
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port        = 0
    to_port          = 0
    protocol         = "-1"
    cidr_blocks      = ["0.0.0.0/0"]
    ipv6_cidr_blocks = ["::/0"]
  }

  tags = {
    Name = "allow_tls"
  }
}

# resource "aws_security_group" "allow_ssh" {
#   name        = "allow_default_ssh"
#   description = "Allow ssh inbound traffic"
#   vpc_id      = module.vpc.id

#   ingress {
#     description = "SSH from VPC"
#     from_port   = 22
#     to_port     = 22
#     protocol    = "tcp"
#     cidr_blocks = [module.vpc.cidr_block, var.myIp]
#   }

#   egress {
#     from_port        = 0
#     to_port          = 0
#     protocol         = "-1"
#     cidr_blocks      = ["0.0.0.0/0"]
#     ipv6_cidr_blocks = ["::/0"]
#   }

#   tags = {
#     Name = "allow_ssh"
#   }
# }

# resource "aws_security_group" "allow_unrestricted_ssh" {
#   name        = "allow_unrestricted_ssh"
#   description = "Allow unrestricted ssh inbound traffic"
#   vpc_id      = module.vpc.id

#   ingress {
#     description = "Unrestricted ssh access"
#     from_port   = 22
#     to_port     = 22
#     protocol    = "tcp"
#     cidr_blocks = ["0.0.0.0/0"]
#   }

#   egress {
#     from_port        = 0
#     to_port          = 0
#     protocol         = "-1"
#     cidr_blocks      = ["0.0.0.0/0"]
#     ipv6_cidr_blocks = ["::/0"]
#   }

#   tags = {
#     Name = "allow_unrestricted_ssh"
#   }
# }

resource "aws_instance" "instance1" {
  ami           = data.aws_ami.amazon2023.id
  instance_type = "t3.micro"
  subnet_id     = module.vpc.public_subnets[0]
  associate_public_ip_address = true

  metadata_options {
    http_tokens = "required"
  }

  tags = {
    Name = "exampleIDMSv2-enabled-${random_id.random.id}"
    Foo  = "bar"
  }
}

resource "aws_instance" "instance2" {
  ami                         = data.aws_ami.amazon2023.id
  instance_type               = "t3.micro"
  subnet_id                   = module.vpc.public_subnets[0]
  associate_public_ip_address = true
  key_name                    = "scottford"

  vpc_security_group_ids = [module.allow_unrestricted_ssh.security_group_id]

  metadata_options {
    http_endpoint = "disabled"
  }

  tags = {
    Name = "exampleIDMSv2-disabled-${random_id.random.id}"
    Foo  = "bar"
  }
}

resource "aws_instance" "instance3" {
  ami           = data.aws_ami.ubuntu2204.id
  instance_type = "t3.micro"
  subnet_id     = module.vpc.public_subnets[0]
  associate_public_ip_address = true 
  key_name = "scottford"

  vpc_security_group_ids = [module.allow_unrestricted_ssh.security_group_id]

  metadata_options {
    http_endpoint = "disabled"
  }

    # root disk
  root_block_device {
    volume_size           = "20"
    volume_type           = "gp2"
    encrypted             = true
    delete_on_termination = true
  }
  # data disk
  ebs_block_device {
    device_name           = "/dev/xvda"
    volume_size           = "50"
    volume_type           = "gp2"
    encrypted             = true
    delete_on_termination = true
  }

  tags = {
    Name = "exampleIDMSv2-not-configured-${random_id.random.id}"
    Foo  = "baz"
  }
}

1. Install cnspec v9
2. Ensure no plugins are installed
3. create a variant policy (example above is valid) called my-variant-policy.mql.yaml and save the file
4. create a main.tf file with terraform hcl
5. execute scan cnspec scan terraform main.tf -f my-variant-policy.mql.yaml

Expected behavior cnspec should download all dependant plugins

[chris-rock]

the solution is not to download all providers. Instead we need to improve the filter so that queries that are not-applicable are filtered out. This would be an extension to https://github.com/mondoohq/cnspec/pull/741

[vjeffrey]

is this fixed now?