`key?' errors on auto generated test scripts

[ezrover]

Hi, I am using: echo "generating inspec script to test terraform infrastructure" inspec terraform generate --overwrite --name setup_profile inspec exec setup_profile -t aws://$REGION

But always face the following types of errors on auto generated tests. Any ideas on how I can eliminate these `key?' errors?

✔ aws_iam_group::swifty-microservice-microservice-dev-committers: InSpec-Iggy aws_iam_group::swifty-microservice-microservice-dev-committers ✔ IAM Group swifty-microservice-microservice-dev-committers should exist × aws_iam_policy::arn:aws:iam::acctno:policy/swifty-microservice/swifty-microservice-microservice-dev-pipeline-checkout-policy: InSpec-Iggy aws_iam_policy::arn:aws:iam::acctno:policy/swifty-microservice/swifty-microservice-microservice-dev-pipeline-checkout-policy (3 failed) × Policy arn:aws:iam::acctno:policy/swifty-microservice/swifty-microservice-microservice-dev-pipeline-checkout-policy should exist expected Policy arn:aws:iam::acctno:policy/swifty-microservice/swifty-microservice-microservice-dev-pipeline-checkout-policy to exist × Policy arn:aws:iam::acctno:policy/swifty-microservice/swifty-microservice-microservice-dev-pipeline-checkout-policy arn should eq "arn:aws:iam::acctno:policy/swifty-microservice/swifty-microservice-microservice-dev-pipeline-checkout-policy"

 expected: "arn:aws:iam::acctno:policy/swifty-microservice/swifty-microservice-microservice-dev-pipeline-checkout-policy"
      got: nil

 (compared using ==)

 ×  Policy arn:aws:iam::acctno:policy/swifty-microservice/swifty-microservice-microservice-dev-pipeline-checkout-policy policy should eq "{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n    {\n      \"Effect\": \"Allow\",\n      \"...redential\"\n      ],\n      \"Resource\": \"arn:aws:iam::*:user/${aws:username}\"\n    }\n  ]\n}\n"

 expected: "{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n    {\n      \"Effect\": \"Allow\",\n      \"...redential\"\n      ],\n      \"Resource\": \"arn:aws:iam::*:user/${aws:username}\"\n    }\n  ]\n}\n"
      got: nil

 (compared using ==)

× aws_iam_policy::arn:aws:iam::acctno:policy/terraform/r2race-swifty-microservice-dev-terraform-ddb-lock-policy: InSpec-Iggy aws_iam_policy::arn:aws:iam::acctno:policy/terraform/r2race-swifty-microservice-dev-terraform-ddb-lock-policy (3 failed) × Policy arn:aws:iam::acctno:policy/terraform/r2race-swifty-microservice-dev-terraform-ddb-lock-policy should exist expected Policy arn:aws:iam::acctno:policy/terraform/r2race-swifty-microservice-dev-terraform-ddb-lock-policy to exist × Policy arn:aws:iam::acctno:policy/terraform/r2race-swifty-microservice-dev-terraform-ddb-lock-policy arn should eq "arn:aws:iam::acctno:policy/terraform/r2race-swifty-microservice-dev-terraform-ddb-lock-policy"

 expected: "arn:aws:iam::acctno:policy/terraform/r2race-swifty-microservice-dev-terraform-ddb-lock-policy"
      got: nil

 (compared using ==)

 ×  Policy arn:aws:iam::acctno:policy/terraform/r2race-swifty-microservice-dev-terraform-ddb-lock-policy policy should eq "{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n    {\n      \"Sid\": \"\",\n      \"Effect\"...odb:us-west-2:acctno:table/r2race-swifty-microservice-dev-terraform-ddb-lock\"\n    }\n  ]\n}"

 expected: "{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n    {\n      \"Sid\": \"\",\n      \"Effect\"...odb:us-west-2:acctno:table/r2race-swifty-microservice-dev-terraform-ddb-lock\"\n    }\n  ]\n}"
      got: nil

 (compared using ==)

✔ aws_iam_role::terraform-state-role: InSpec-Iggy aws_iam_role::terraform-state-role ✔ IAM Role terraform-state-role should exist ✔ IAM Role terraform-state-role description should eq "This role gives Terraform users permissions to manage infrastructure" ✔ aws_iam_user::swifty-developer: InSpec-Iggy aws_iam_user::swifty-developer ✔ IAM User swifty-developer should exist ✔ IAM User swifty-developer name should eq "swifty-developer" ✔ aws_s3_bucket::swifty-microservice-microservice-dev-logs: InSpec-Iggy aws_s3_bucket::swifty-microservice-microservice-dev-logs ✔ S3 Bucket swifty-microservice-microservice-dev-logs should exist ✔ S3 Bucket swifty-microservice-microservice-dev-logs region should eq "us-west-2" ✔ aws_s3_bucket::r2race-swifty-microservice-dev-terraform-s3-bucket: InSpec-Iggy aws_s3_bucket::r2race-swifty-microservice-dev-terraform-s3-bucket ✔ S3 Bucket r2race-swifty-microservice-dev-terraform-s3-bucket should exist ✔ S3 Bucket r2race-swifty-microservice-dev-terraform-s3-bucket region should eq "us-west-2" ✔ aws_sns_topic::arn:aws:sns:us-west-2:acctno:devops-emailalert-sns-topic: InSpec-Iggy aws_sns_topic::arn:aws:sns:us-west-2:acctno:devops-emailalert-sns-topic ✔ aws_sns_topic should exist ✔ aws_sns_topic arn should eq "arn:aws:sns:us-west-2:acctno:devops-emailalert-sns-topic" ✔ aws_sns_topic::arn:aws:sns:us-west-2:acctno:devops-slack-alerts-sns-topic: InSpec-Iggy aws_sns_topic::arn:aws:sns:us-west-2:acctno:devops-slack-alerts-sns-topic ✔ aws_sns_topic should exist ✔ aws_sns_topic arn should eq "arn:aws:sns:us-west-2:acctno:devops-slack-alerts-sns-topic" × aws_cloudwatch_log_metric_filter::swifty-microservice-microservice-dev-notify-slack-errors: InSpec-Iggy aws_cloudwatch_log_metric_filter::swifty-microservice-microservice-dev-notify-slack-errors × Control Source Code Error setup_profile/controls/controls.rb:123 undefined method `key?' for "swifty-microservice-microservice-dev-notify-slack-errors":String × aws_iam_policy::arn:aws:iam::acctno:policy/swifty-microservice-microservice-dev-notify-slack-lambda-policy: InSpec-Iggy aws_iam_policy::arn:aws:iam::acctno:policy/swifty-microservice-microservice-dev-notify-slack-lambda-policy (3 failed) × Policy arn:aws:iam::acctno:policy/swifty-microservice-microservice-dev-notify-slack-lambda-policy should exist expected Policy arn:aws:iam::acctno:policy/swifty-microservice-microservice-dev-notify-slack-lambda-policy to exist × Policy arn:aws:iam::acctno:policy/swifty-microservice-microservice-dev-notify-slack-lambda-policy arn should eq "arn:aws:iam::acctno:policy/swifty-microservice-microservice-dev-notify-slack-lambda-policy"

 expected: "arn:aws:iam::acctno:policy/swifty-microservice-microservice-dev-notify-slack-lambda-policy"
      got: nil

 (compared using ==)

 ×  Policy arn:aws:iam::acctno:policy/swifty-microservice-microservice-dev-notify-slack-lambda-policy policy should eq "{\n    \"Version\": \"2012-10-17\",\n    \"Statement\": [\n      {\n        \"Sid\": \"AllowWriteToC..."xray:GetSamplingStatisticSummaries\"\n        ],\n        \"Resource\": \"*\"\n      }\n    ]\n}\n"

 expected: "{\n    \"Version\": \"2012-10-17\",\n    \"Statement\": [\n      {\n        \"Sid\": \"AllowWriteToC..."xray:GetSamplingStatisticSummaries\"\n        ],\n        \"Resource\": \"*\"\n      }\n    ]\n}\n"
      got: nil

 (compared using ==)

✔ aws_iam_role::swifty-microservice-microservice-dev-notify-slack-lambda-role: InSpec-Iggy aws_iam_role::swifty-microservice-microservice-dev-notify-slack-lambda-role ✔ IAM Role swifty-microservice-microservice-dev-notify-slack-lambda-role should exist ✔ IAM Role swifty-microservice-microservice-dev-notify-slack-lambda-role description should eq "notify-slack-lambda-role"

[mattray]

I believe this was caused by a mismatch between InSpec's resources and the Terraform ones. It's supposed to skip things that don't map cleanly. If you could verify this still recreates, perhaps with the most recent InSpec release I'll see if I can prevent these from happening. If you can post the content of the controls.rb or the output as you run it with --debug that would also be useful.

[mattray]

This appears in the Azure resources, I'll fix those in 0.5.0 but I'll have to come up with a better mapping system for resource packs and resources that take a key in the next release.