terraform-google-modules / terraform-google-vpc-service-controls

Handles opinionated VPC Service Controls and Access Context Manager configuration and deployments
https://registry.terraform.io/modules/terraform-google-modules/vpc-service-controls/google
Apache License 2.0
61 stars 70 forks source link

Negative tests fail on master #6

Closed ivankorn closed 4 years ago

ivankorn commented 5 years ago

negative tests fail on master branch (tf11)

kitchen create

bash-4.4# kitchen create
-----> Starting Kitchen (v1.23.5)
$$$$$$ Running command `terraform version` in directory /cft/workdir
       Terraform v0.11.13

       Your version of Terraform is out of date! The latest version
       is 0.12.5. You can update by downloading from www.terraform.io/downloads.html
$$$$$$ Terraform v0.11.13 is supported
-----> Creating <simple-example-local>...
$$$$$$ Running command `terraform init -input=false -lock=true -lock-timeout=0s  -upgrade -force-copy -backend=true  -get=true -get-plugins=true  -verify-plugins=true` in directory /cft/workdir/test/fixtures/simple_example
       Upgrading modules...
       - module.example
         Updating source "../../../examples/simple_example"
       - module.example.access_context_manager_policy
         Updating source "../.."
       - module.example.access_level_members
         Updating source "../../modules/access_level"
       - module.example.regular_service_perimeter_1
         Updating source "../../modules/regular_service_perimeter"
       - module.example.bigquery
         Found version 0.1.0 of terraform-google-modules/bigquery/google on registry.terraform.io
         Updating source "terraform-google-modules/bigquery/google"

       Initializing provider plugins...
       - Checking for available provider plugins on https://releases.hashicorp.com...
       - Downloading plugin for provider "google" (2.5.1)...

       Terraform has been successfully initialized!
$$$$$$ Running command `terraform workspace select kitchen-terraform-simple-example-local` in directory /cft/workdir/test/fixtures/simple_example

       Workspace "kitchen-terraform-simple-example-local" doesn't exist.

       You can create this workspace with the "new" subcommand.
$$$$$$ Running command `terraform workspace new kitchen-terraform-simple-example-local` in directory /cft/workdir/test/fixtures/simple_example
       Created and switched to workspace "kitchen-terraform-simple-example-local"!

       You're now on a new, empty workspace. Workspaces isolate their state,
       so if you run "terraform plan" Terraform will not see any existing state
       for this configuration.
       Finished creating <simple-example-local> (0m34.08s).
-----> Kitchen is finished. (0m35.40s)

known issue - credentials file in docker

bash-4.4# ls credentials.json credentials.json bash-4.4# cat credentials.json { "type": "service_account", "project_id": "gl-ivankorniienko-seed", "private_key_id": ... ... "client_email": "project-factory-9942@gl-ivankorniienko-seed.iam.gserviceaccount.com", "client_id": "112315488975309390051", "auth_uri": "https://accounts.google.com/o/oauth2/auth", "token_uri": "https://oauth2.googleapis.com/token", "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs", "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/project-factory-9942%40gl-ivankorniienko-seed.iam.gserviceaccount.com" } bash-4.4# env RUBYGEMS_VERSION=2.7.8 BUNDLER_VERSION=1.17.1 HOSTNAME=960652f7d941 RUBY_VERSION=2.5.3 GEM_HOME=/usr/local/bundle GOOGLE_APPLICATION_CREDENTIALS=/tmp/tmp.EUCRbM0rnZ CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE=/tmp/tmp.EUCRbM0rnZ APP_BASE_DIR=/cft PWD=/cft/workdir TF_VAR_bucket_name= HOME=/cft/home BUNDLE_PATH=/usr/local/bundle BUNDLE_APP_CONFIG=/usr/local/bundle TF_VAR_project_id= TERM=xterm BUNDLE_SILENCE_ROOT_WARNING=1 RUBY_MAJOR=2.5 SHLVL=1 RUBY_DOWNLOADSHA256=1cc9d0359a8ea35fc6111ec830d12e60168f3b9b305a3c2578357d360fcf306f PATH=/cft/bin:/cft/google-cloud-sdk/bin:/usr/local/bundle/bin:/usr/local/bundle/gems/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin =/usr/bin/env

bash-4.4# cp credentials.json /tmp/tmp.EUCRbM0rnZ

kitchen converge

bash-4.4# kitchen converge
-----> Starting Kitchen (v1.23.5)
$$$$$$ Running command `terraform version` in directory /cft/workdir
       Terraform v0.11.13

       Your version of Terraform is out of date! The latest version
       is 0.12.5. You can update by downloading from www.terraform.io/downloads.html
$$$$$$ Terraform v0.11.13 is supported
-----> Converging <simple-example-local>...
$$$$$$ Running command `terraform workspace select kitchen-terraform-simple-example-local` in directory /cft/workdir/test/fixtures/simple_example
$$$$$$ Running command `terraform get -update` in directory /cft/workdir/test/fixtures/simple_example
       - module.example
         Updating source "../../../examples/simple_example"
       - module.example.access_context_manager_policy
         Updating source "../.."
       - module.example.access_level_members
         Updating source "../../modules/access_level"
       - module.example.regular_service_perimeter_1
         Updating source "../../modules/regular_service_perimeter"
       - module.example.bigquery
         Found version 0.1.0 of terraform-google-modules/bigquery/google on registry.terraform.io
         Updating source "terraform-google-modules/bigquery/google"
$$$$$$ Running command `terraform validate -check-variables=true   ` in directory /cft/workdir/test/fixtures/simple_example
$$$$$$ Running command `terraform apply -lock=true -lock-timeout=0s -input=false -auto-approve=true  -parallelism=10 -refresh=true  ` in directory /cft/workdir/test/fixtures/simple_example
       module.example.module.bigquery.google_bigquery_dataset.main: Creating...
         access.#:                    "" => "<computed>"
         creation_time:               "" => "<computed>"
         dataset_id:                  "" => "sample_dataset"
         default_table_expiration_ms: "" => "3600000"
         description:                 "" => "Dataset with a single table with one field"
         etag:                        "" => "<computed>"
         friendly_name:               "" => "sample_dataset"
         labels.%:                    "" => "3"
         labels.billable:             "" => "true"
         labels.env:                  "" => "dev"
         labels.owner:                "" => "janesmith"
         last_modified_time:          "" => "<computed>"
         location:                    "" => "US"
         project:                     "" => "gl-ivankorniienko-seed"
         self_link:                   "" => "<computed>"
       module.example.module.access_context_manager_policy.google_access_context_manager_access_policy.access_policy: Creating...
         create_time: "" => "<computed>"
         name:        "" => "<computed>"
         parent:      "" => "organizations/826592752744"
         title:       "" => "ik_test_policy"
         update_time: "" => "<computed>"
       module.example.module.bigquery.google_bigquery_dataset.main: Creation complete after 2s (ID: gl-ivankorniienko-seed:sample_dataset)
       module.example.module.bigquery.google_bigquery_table.main: Creating...
         creation_time:            "" => "<computed>"
         dataset_id:               "" => "sample_dataset"
         etag:                     "" => "<computed>"
         expiration_time:          "" => "<computed>"
         labels.%:                 "" => "3"
         labels.billable:          "" => "true"
         labels.env:               "" => "dev"
         labels.owner:             "" => "joedoe"
         last_modified_time:       "" => "<computed>"
         location:                 "" => "<computed>"
         num_bytes:                "" => "<computed>"
         num_long_term_bytes:      "" => "<computed>"
         num_rows:                 "" => "<computed>"
         project:                  "" => "gl-ivankorniienko-seed"
         schema:                   "" => "[{\"description\":\"Example Field\",\"mode\":\"NULLABLE\",\"name\":\"name\",\"type\":\"STRING\"}]"
         self_link:                "" => "<computed>"
         table_id:                 "" => "example_table"
         time_partitioning.#:      "" => "1"
         time_partitioning.0.type: "" => "DAY"
         type:                     "" => "<computed>"
       module.example.module.bigquery.google_bigquery_table.main: Creation complete after 0s (ID: gl-ivankorniienko-seed:sample_dataset.example_table)
       module.example.module.access_context_manager_policy.google_access_context_manager_access_policy.access_policy: Creation complete after 7s (ID: 434978528352)
       module.example.module.regular_service_perimeter_1.google_access_context_manager_service_perimeter.regular_service_perimeter: Creating...
         create_time:                    "" => "<computed>"
         name:                           "" => "accessPolicies/434978528352/servicePerimeters/regular_perimeter_1"
         parent:                         "" => "accessPolicies/434978528352"
         perimeter_type:                 "" => "PERIMETER_TYPE_REGULAR"
         status.#:                       "" => "1"
         status.0.access_levels.#:       "" => "1"
         status.0.access_levels.0:       "" => "accessPolicies/434978528352/accessLevels/terraform_members"
         status.0.resources.#:           "" => "1"
         status.0.resources.0:           "" => "projects/559486914361"
         status.0.restricted_services.#: "" => "2"
         status.0.restricted_services.0: "" => "bigquery.googleapis.com"
         status.0.restricted_services.1: "" => "storage.googleapis.com"
         title:                          "" => "regular_perimeter_1"
         update_time:                    "" => "<computed>"
       module.example.module.access_level_members.google_access_context_manager_access_level.access_level: Creating...
         basic.#:                                                  "" => "1"
         basic.0.combining_function:                               "" => "AND"
         basic.0.conditions.#:                                     "" => "1"
         basic.0.conditions.0.device_policy.#:                     "" => "1"
         basic.0.conditions.0.device_policy.0.os_constraints.#:    "" => "1"
         basic.0.conditions.0.device_policy.0.require_screen_lock: "" => "false"
         basic.0.conditions.0.members.#:                           "" => "2"
         basic.0.conditions.0.members.0:                           "" => "user:ivan@phoogle.net"
         basic.0.conditions.0.members.1:                           "" => "serviceAccount:559486914361-compute@developer.gserviceaccount.com"
         basic.0.conditions.0.negate:                              "" => "false"
         name:                                                     "" => "accessPolicies/434978528352/accessLevels/terraform_members"
         parent:                                                   "" => "accessPolicies/434978528352"
         title:                                                    "" => "terraform_members"
       module.example.module.access_level_members.google_access_context_manager_access_level.access_level: Creation complete after 4s (ID: accessPolicies/434978528352/accessLevels/terraform_members)

       Error: Error applying plan:

       1 error(s) occurred:

       * module.example.module.regular_service_perimeter_1.google_access_context_manager_service_perimeter.regular_service_perimeter: 1 error(s) occurred:

       * google_access_context_manager_service_perimeter.regular_service_perimeter: Error creating ServicePerimeter: googleapi: Error 400: Level name 'terraform_members' is not available in this Access Policy resource, but is referenced in 'status' field of Perimeter 'regular_perimeter_1'. If you are trying to delete a Level which is referenced in this Perimeter, you must first remove the reference.

       Terraform does not automatically rollback in the face of errors.
       Instead, your Terraform state file has been partially updated with
       any resources that successfully completed. Please address the error
       above and apply again to incrementally change your infrastructure.

>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 1 actions failed.
>>>>>>     Converge failed on instance <simple-example-local>.  Please see .kitchen/logs/simple-example-local.log for more details
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

known issue - 2*kitchen converge

bash-4.4# kitchen converge 
.git/                            .ruby-version                    Makefile                         credentials.json.ikorniienko     main.tf                          v0.1.0                           
.gitignore                       CHANGELOG.md                     README.md                        credentials.json_ivankorniienko  modules/                         variables.tf                     
.kitchen/                        Gemfile                          converge_err.txt                 examples/                        outputs.tf                       
.kitchen.yml                     LICENSE                          credentials.json                 helpers/                         test/                            
bash-4.4# kitchen list
$$$$$$ Running command `terraform version` in directory /cft/workdir
       Terraform v0.11.13

       Your version of Terraform is out of date! The latest version
       is 0.12.5. You can update by downloading from www.terraform.io/downloads.html
$$$$$$ Terraform v0.11.13 is supported
^[[AInstance              Driver     Provisioner  Verifier   Transport  Last Action  Last Error
simple-example-local  Terraform  Terraform    Terraform  Ssh        Created      Kitchen::ActionFailed
bash-4.4# kitchen converge simple-example-local
-----> Starting Kitchen (v1.23.5)
$$$$$$ Running command `terraform version` in directory /cft/workdir
       Terraform v0.11.13

       Your version of Terraform is out of date! The latest version
       is 0.12.5. You can update by downloading from www.terraform.io/downloads.html
$$$$$$ Terraform v0.11.13 is supported
-----> Converging <simple-example-local>...
$$$$$$ Running command `terraform workspace select kitchen-terraform-simple-example-local` in directory /cft/workdir/test/fixtures/simple_example
$$$$$$ Running command `terraform get -update` in directory /cft/workdir/test/fixtures/simple_example
       - module.example
         Updating source "../../../examples/simple_example"
       - module.example.access_context_manager_policy
         Updating source "../.."
       - module.example.access_level_members
         Updating source "../../modules/access_level"
       - module.example.regular_service_perimeter_1
         Updating source "../../modules/regular_service_perimeter"
       - module.example.bigquery
         Found version 0.1.0 of terraform-google-modules/bigquery/google on registry.terraform.io
         Updating source "terraform-google-modules/bigquery/google"
$$$$$$ Running command `terraform validate -check-variables=true   ` in directory /cft/workdir/test/fixtures/simple_example
$$$$$$ Running command `terraform apply -lock=true -lock-timeout=0s -input=false -auto-approve=true  -parallelism=10 -refresh=true  ` in directory /cft/workdir/test/fixtures/simple_example
       google_access_context_manager_access_policy.access_policy: Refreshing state... (ID: 434978528352)
       google_bigquery_dataset.main: Refreshing state... (ID: gl-ivankorniienko-seed:sample_dataset)
       google_bigquery_table.main: Refreshing state... (ID: gl-ivankorniienko-seed:sample_dataset.example_table)
       google_access_context_manager_access_level.access_level: Refreshing state... (ID: accessPolicies/434978528352/accessLevels/terraform_members)
       module.example.module.regular_service_perimeter_1.google_access_context_manager_service_perimeter.regular_service_perimeter: Creating...
         create_time:                    "" => "<computed>"
         name:                           "" => "accessPolicies/434978528352/servicePerimeters/regular_perimeter_1"
         parent:                         "" => "accessPolicies/434978528352"
         perimeter_type:                 "" => "PERIMETER_TYPE_REGULAR"
         status.#:                       "" => "1"
         status.0.access_levels.#:       "" => "1"
         status.0.access_levels.0:       "" => "accessPolicies/434978528352/accessLevels/terraform_members"
         status.0.resources.#:           "" => "1"
         status.0.resources.0:           "" => "projects/559486914361"
         status.0.restricted_services.#: "" => "2"
         status.0.restricted_services.0: "" => "bigquery.googleapis.com"
         status.0.restricted_services.1: "" => "storage.googleapis.com"
         title:                          "" => "regular_perimeter_1"
         update_time:                    "" => "<computed>"
       module.example.module.access_level_members.google_access_context_manager_access_level.access_level: Modifying... (ID: accessPolicies/434978528352/accessLevels/terraform_members)
         basic.0.conditions.0.device_policy.#:                     "0" => "1"
         basic.0.conditions.0.device_policy.0.os_constraints.#:    "0" => "1"
         basic.0.conditions.0.device_policy.0.require_screen_lock: "" => "false"
       module.example.module.regular_service_perimeter_1.google_access_context_manager_service_perimeter.regular_service_perimeter: Creation complete after 3s (ID: accessPolicies/434978528352/servicePerimeters/regular_perimeter_1)
       module.example.module.access_level_members.google_access_context_manager_access_level.access_level: Modifications complete after 4s (ID: accessPolicies/434978528352/accessLevels/terraform_members)

       Apply complete! Resources: 1 added, 1 changed, 0 destroyed.

       Outputs:

       dataset_id = gl-ivankorniienko-seed:sample_dataset
       parent_id = 826592752744
       policy_name = ik_test_policy
       protected_project_id = gl-ivankorniienko-seed
       public_project_id = gl-ik-test
       table_id = gl-ivankorniienko-seed:sample_dataset.example_table
       Finished converging <simple-example-local> (0m9.57s).
-----> Kitchen is finished. (0m10.85s)

kitchen verify

bash-4.4# kitchen verify
-----> Starting Kitchen (v1.23.5)
$$$$$$ Running command `terraform version` in directory /cft/workdir
       Terraform v0.11.13

       Your version of Terraform is out of date! The latest version
       is 0.12.5. You can update by downloading from www.terraform.io/downloads.html
$$$$$$ Terraform v0.11.13 is supported
-----> Setting up <simple-example-local>...
       Finished setting up <simple-example-local> (0m0.00s).
-----> Verifying <simple-example-local>...
$$$$$$ Running command `terraform output -json` in directory /cft/workdir/test/fixtures/simple_example
       {
           "dataset_id": {
        "sensitive": false,
        "type": "string",
        "value": "gl-ivankorniienko-seed:sample_dataset"
           },
           "parent_id": {
        "sensitive": false,
        "type": "string",
        "value": "826592752744"
           },
           "policy_name": {
        "sensitive": false,
        "type": "string",
        "value": "ik_test_policy"
           },
           "protected_project_id": {
        "sensitive": false,
        "type": "string",
        "value": "gl-ivankorniienko-seed"
           },
           "public_project_id": {
        "sensitive": false,
        "type": "string",
        "value": "gl-ik-test"
           },
           "table_id": {
        "sensitive": false,
        "type": "string",
        "value": "gl-ivankorniienko-seed:sample_dataset.example_table"
           }
       }
Verifying simple_example
Skipping profile: 'inspec-gcp' on unsupported platform: 'alpine/3.8.1'.

Profile: simple_example
Version: (not specified)
Target:  local://

  ✔  big_query_vpc_positive_test: Command: `bq query --use_legacy=false --project_id=gl-ivankorniienko-seed 'select * from `gl-ivankorniienko-seed.sample_dataset.example_table` limit 10'`
     ✔  Command: `bq query --use_legacy=false --project_id=gl-ivankorniienko-seed 'select * from `gl-ivankorniienko-seed.sample_dataset.example_table` limit 10'` exit_status should equal 0
     ✔  Command: `bq query --use_legacy=false --project_id=gl-ivankorniienko-seed 'select * from `gl-ivankorniienko-seed.sample_dataset.example_table` limit 10'` stderr should include "Current status: DONE"
  ×  big_query_vpc_negative_test: Command: `bq query --use_legacy=false --project_id=gl-ik-test 'select * from `gl-ivankorniienko-seed.sample_dataset.example_table` limit 10'` (3 failed)
     ×  Command: `bq query --use_legacy=false --project_id=gl-ik-test 'select * from `gl-ivankorniienko-seed.sample_dataset.example_table` limit 10'` exit_status should equal 1

     expected #<Integer:3> => 1
          got #<Integer:1> => 0

     Compared using equal?, which compares object identity,
     but expected and actual are not the same object. Use
     `expect(actual).to eq(expected)` if you don't care about
     object identity in this example.

     ×  Command: `bq query --use_legacy=false --project_id=gl-ik-test 'select * from `gl-ivankorniienko-seed.sample_dataset.example_table` limit 10'` stderr should eq ""

     expected: ""
          got: "\rWaiting on bqjob_r6885113e6bc5ded0_0000016c24564555_1 ... (0s) Current status: RUNNING\r          ...           \rWaiting on bqjob_r6885113e6bc5ded0_0000016c24564555_1 ... (0s) Current status: DONE   "

     (compared using ==)

     ×  Command: `bq query --use_legacy=false --project_id=gl-ik-test 'select * from `gl-ivankorniienko-seed.sample_dataset.example_table` limit 10'` stdout should include "Request is prohibited by organization's policy."
     expected "\n" to include "Request is prohibited by organization's policy."
     Diff:
     @@ -1,2 +1 @@
     -Request is prohibited by organization's policy.

Profile Summary: 1 successful control, 1 control failure, 0 controls skipped
Test Summary: 2 successful, 3 failures, 0 skipped
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 1 actions failed.
>>>>>>     Verify failed on instance <simple-example-local>.  Please see .kitchen/logs/simple-example-local.log for more details
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

bash-4.4# bq query --use_legacy=false --project_id=gl-ivankorniienko-seed 'select * from `gl-ivankorniienko-seed.sample_dataset.example_table` limit 10'
Waiting on bqjob_r7706390d626f1f1c_0000016c245779ea_1 ... (0s) Current status: DONE   
bash-4.4# 
bash-4.4# bq query --use_legacy=false --project_id=gl-ik-test 'select * from `gl-ivankorniienko-seed.sample_dataset.example_table` limit 10'
Waiting on bqjob_r3e7599b738cd0243_0000016c245a649a_1 ... (0s) Current status: DONE   
bash-4.4# echo $?
0

kitchen log:

bash-4.4# cat .kitchen/logs/kitchen.log
I, [2019-07-24T14:34:04.219535 #2784]  INFO -- Kitchen: -----> Starting Kitchen (v1.23.5)
W, [2019-07-24T14:34:04.375168 #2784]  WARN -- Kitchen: Running command `terraform version` in directory /cft/workdir
I, [2019-07-24T14:34:04.390765 #2784]  INFO -- Kitchen: Terraform v0.11.13
I, [2019-07-24T14:34:04.390992 #2784]  INFO -- Kitchen: 
I, [2019-07-24T14:34:04.391063 #2784]  INFO -- Kitchen: Your version of Terraform is out of date! The latest version
I, [2019-07-24T14:34:04.391129 #2784]  INFO -- Kitchen: is 0.12.5. You can update by downloading from www.terraform.io/downloads.html
W, [2019-07-24T14:34:04.403463 #2784]  WARN -- Kitchen: Terraform v0.11.13 is supported
I, [2019-07-24T14:34:05.531096 #2784]  INFO -- Kitchen: -----> Setting up <simple-example-local>...
I, [2019-07-24T14:34:05.531701 #2784]  INFO -- Kitchen: -----> Verifying <simple-example-local>...
E, [2019-07-24T14:34:14.136741 #2784] ERROR -- Kitchen: ------Exception-------
E, [2019-07-24T14:34:14.136801 #2784] ERROR -- Kitchen: Class: Kitchen::ActionFailed
E, [2019-07-24T14:34:14.136828 #2784] ERROR -- Kitchen: Message: 1 actions failed.
>>>>>>     Verify failed on instance <simple-example-local>.  Please see .kitchen/logs/simple-example-local.log for more details
E, [2019-07-24T14:34:14.136855 #2784] ERROR -- Kitchen: ----------------------
E, [2019-07-24T14:34:14.136878 #2784] ERROR -- Kitchen: ------Backtrace-------
E, [2019-07-24T14:34:14.136899 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/command.rb:183:in `report_errors'
E, [2019-07-24T14:34:14.136923 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/command.rb:174:in `run_action'
E, [2019-07-24T14:34:14.136944 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/command/action.rb:36:in `block in call'
E, [2019-07-24T14:34:14.136968 #2784] ERROR -- Kitchen: /usr/local/lib/ruby/2.5.0/benchmark.rb:293:in `measure'
E, [2019-07-24T14:34:14.136989 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/command/action.rb:34:in `call'
E, [2019-07-24T14:34:14.137011 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/cli.rb:52:in `perform'
E, [2019-07-24T14:34:14.137034 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/cli.rb:193:in `block (2 levels) in <class:CLI>'
E, [2019-07-24T14:34:14.137056 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/thor-0.20.3/lib/thor/command.rb:27:in `run'
E, [2019-07-24T14:34:14.137088 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/thor-0.20.3/lib/thor/invocation.rb:126:in `invoke_command'
E, [2019-07-24T14:34:14.137122 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/thor-0.20.3/lib/thor.rb:387:in `dispatch'
E, [2019-07-24T14:34:14.137148 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/thor-0.20.3/lib/thor/base.rb:466:in `start'
E, [2019-07-24T14:34:14.137170 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/bin/kitchen:13:in `block in <top (required)>'
E, [2019-07-24T14:34:14.137194 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/errors.rb:171:in `with_friendly_errors'
E, [2019-07-24T14:34:14.137263 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/bin/kitchen:13:in `<top (required)>'
E, [2019-07-24T14:34:14.137292 #2784] ERROR -- Kitchen: /usr/local/bundle/bin/kitchen:23:in `load'
E, [2019-07-24T14:34:14.137315 #2784] ERROR -- Kitchen: /usr/local/bundle/bin/kitchen:23:in `<main>'
E, [2019-07-24T14:34:14.137336 #2784] ERROR -- Kitchen: ----End Backtrace-----
E, [2019-07-24T14:34:14.137357 #2784] ERROR -- Kitchen: -Composite Exception--
E, [2019-07-24T14:34:14.137378 #2784] ERROR -- Kitchen: Class: Kitchen::InstanceFailure
E, [2019-07-24T14:34:14.137399 #2784] ERROR -- Kitchen: Message: Verify failed on instance <simple-example-local>.  Please see .kitchen/logs/simple-example-local.log for more details
E, [2019-07-24T14:34:14.137421 #2784] ERROR -- Kitchen: ----------------------
E, [2019-07-24T14:34:14.137452 #2784] ERROR -- Kitchen: ------Backtrace-------
E, [2019-07-24T14:34:14.137484 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/kitchen-terraform-4.1.0/lib/kitchen/verifier/terraform.rb:91:in `rescue in call'
E, [2019-07-24T14:34:14.137509 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/kitchen-terraform-4.1.0/lib/kitchen/verifier/terraform.rb:85:in `call'
E, [2019-07-24T14:34:14.137529 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/instance.rb:458:in `block in verify_action'
E, [2019-07-24T14:34:14.137558 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/instance.rb:551:in `synchronize_or_call'
E, [2019-07-24T14:34:14.137579 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/instance.rb:513:in `block in action'
E, [2019-07-24T14:34:14.137599 #2784] ERROR -- Kitchen: /usr/local/lib/ruby/2.5.0/benchmark.rb:293:in `measure'
E, [2019-07-24T14:34:14.137617 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/instance.rb:512:in `action'
E, [2019-07-24T14:34:14.137636 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/instance.rb:450:in `verify_action'
E, [2019-07-24T14:34:14.137658 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/instance.rb:382:in `block (2 levels) in transition_to'
E, [2019-07-24T14:34:14.137677 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/lifecycle_hooks.rb:45:in `run_with_hooks'
E, [2019-07-24T14:34:14.137698 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/instance.rb:381:in `block in transition_to'
E, [2019-07-24T14:34:14.137717 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/instance.rb:380:in `each'
E, [2019-07-24T14:34:14.137738 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/instance.rb:380:in `transition_to'
E, [2019-07-24T14:34:14.137757 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/instance.rb:162:in `verify'
E, [2019-07-24T14:34:14.137775 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/command.rb:197:in `public_send'
E, [2019-07-24T14:34:14.137794 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/command.rb:197:in `run_action_in_thread'
E, [2019-07-24T14:34:14.137813 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/command.rb:169:in `block (2 levels) in run_action'
E, [2019-07-24T14:34:14.137835 #2784] ERROR -- Kitchen: ----End Backtrace-----
E, [2019-07-24T14:34:14.137853 #2784] ERROR -- Kitchen: ---Nested Exception---
E, [2019-07-24T14:34:14.137873 #2784] ERROR -- Kitchen: Class: Kitchen::ActionFailed
E, [2019-07-24T14:34:14.137891 #2784] ERROR -- Kitchen: Message: InSpec Runner exited with 1
E, [2019-07-24T14:34:14.137909 #2784] ERROR -- Kitchen: ----------------------
E, [2019-07-24T14:34:14.137928 #2784] ERROR -- Kitchen: ------Backtrace-------
E, [2019-07-24T14:34:14.137946 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/kitchen-terraform-4.1.0/lib/kitchen/verifier/terraform.rb:91:in `rescue in call'
E, [2019-07-24T14:34:14.137966 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/kitchen-terraform-4.1.0/lib/kitchen/verifier/terraform.rb:85:in `call'
E, [2019-07-24T14:34:14.137985 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/instance.rb:458:in `block in verify_action'
E, [2019-07-24T14:34:14.138024 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/instance.rb:551:in `synchronize_or_call'
E, [2019-07-24T14:34:14.138047 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/instance.rb:513:in `block in action'
E, [2019-07-24T14:34:14.138084 #2784] ERROR -- Kitchen: /usr/local/lib/ruby/2.5.0/benchmark.rb:293:in `measure'
E, [2019-07-24T14:34:14.138104 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/instance.rb:512:in `action'
E, [2019-07-24T14:34:14.138141 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/instance.rb:450:in `verify_action'
E, [2019-07-24T14:34:14.138174 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/instance.rb:382:in `block (2 levels) in transition_to'
E, [2019-07-24T14:34:14.138196 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/lifecycle_hooks.rb:45:in `run_with_hooks'
E, [2019-07-24T14:34:14.138218 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/instance.rb:381:in `block in transition_to'
E, [2019-07-24T14:34:14.138241 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/instance.rb:380:in `each'
E, [2019-07-24T14:34:14.138260 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/instance.rb:380:in `transition_to'
E, [2019-07-24T14:34:14.138282 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/instance.rb:162:in `verify'
E, [2019-07-24T14:34:14.138305 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/command.rb:197:in `public_send'
E, [2019-07-24T14:34:14.138326 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/command.rb:197:in `run_action_in_thread'
E, [2019-07-24T14:34:14.138364 #2784] ERROR -- Kitchen: /usr/local/bundle/gems/test-kitchen-1.23.5/lib/kitchen/command.rb:169:in `block (2 levels) in run_action'
E, [2019-07-24T14:34:14.138391 #2784] ERROR -- Kitchen: ----End Backtrace-----
Tfmenard commented 5 years ago

I reproduced the issue only appears if you run kitchen verify within 1 minute after successfully running kitchen converge. So as it is the case for kitchen converge if you re-rerun kitchen verify The negative test pass. I believe this is also related to the propagation delay in VPC SC also mentioned here

What happens when you rerun kitchen verify. Does it always fail?