search

gruntwork-io / terratest

Terratest is a Go library that makes it easier to write automated tests for your infrastructure code.
https://terratest.gruntwork.io/
Apache License 2.0
7.5k stars 1.32k forks source link

Passing list variables from TerraTest go files to Terraform #200

Closed MichaelSmyth0184 closed 5 years ago

MichaelSmyth0184 commented 5 years ago

I have currently been using TerraTest for testing my Terraform IAC. Works fine, but in none of the examples I can see how we would pass a list of values, lets say subnet CIDR blocks or a list of Security group ID's from our Go Terratest file to Terraform, as those values expect a list variable type.

Could you also share any information around the httpHelper script? I have been using your examples to base my test files on, but cannot seem get my script working on my own applications such as Jenkins running on port 8080 which was launched using a gold AMI in aws.

I can navigate to the Application on my browser, and curl information from it from my machine where I am running the tests, but still does not seem to work.

There is also logs when I run the test showing the HTML layout, but I think it might be something to do with the InstanceText in which I am passing to the test which makes it fail.

http_helper.HttpGetWithRetry(t, instanceURL, 200, instanceText, maxRetries, timeBetweenRetries)

Thanks,

Michael

brikis98 commented 5 years ago

Works fine, but in none of the examples I can see how we would pass a list of values, lets say subnet CIDR blocks or a list of Security group ID's from our Go Terratest file to Terraform, as those values expect a list variable type.

Just pass a list :)

myList := []string{"foo", "bar", "baz"}

terraformOptions := &terraform.Options{
  // The path to where our Terraform code is located
  TerraformDir: "../my-module",

  // Variables to pass to our Terraform code using -var options
  Vars: map[string]interface{}{
    "my_list": myList,
  },
}

I have been using your examples to base my test files on, but cannot seem get my script working on my own applications such as Jenkins running on port 8080 which was launched using a gold AMI in aws.

What error do you get? Can you share your log output?

MichaelSmyth0184 commented 5 years ago

Hi,

I think it might be something to do with the body of the http GET as I am trying to test if my sonarqube is up and working and I do get a Response status: 200 along with all the HTML of sonar, but the test stills fails.

TestPrestaHttpResponse`` 2018-11-28T11:44:55Z retry.go:69: HTTP GET to URL http://3.121.38.141:9000 TestPrestaHttpResponse 2018-11-28T11:44:55Z` http_helper.go:27: Making an HTTP GET call to URL http://3.121.38.141:9000 TestPrestaHttpResponse 2018-11-28T11:44:55Z retry.go:81: HTTP GET to URL http://3.121.38.141:9000 returned an error: Validation failed for URL http://3.121.38.141:9000. Response staus: 200. Response body: <!DOCTYPE html><meta http-equiv="content-type" content="text/html; charset=UTF-8charset="UTF-8"/>

The example on the Terratest github with the basic HTML webpage saying "Hello, World!" works grand as it has a basic with some text! But for other applications where the HTML has alot going on I cannot seem to get the test to Pass.

brikis98 commented 5 years ago

If you need custom validation, use HttpGetWithRetryWithCustomValidation instead. It lets you pass in a function to check the status code and body of the response any way you want (e.g., via strings.Contains instead of simple equality).

MichaelSmyth0184 commented 5 years ago

@brikis98 Thank you, got both working now!!!

Many thanks,

Michael

brikis98 commented 5 years ago

Great to hear!

MatthiasScholz commented 3 years ago

Sorry for opening up this conversation, but just for completeness it seems this issue is correlated with:

Just recently I ran into this issue trying to write a test using a list as input. Maybe the connection is helpful for others.

rishabhanand26 commented 3 years ago

i tried passing a list to my variable like this :

Vars: map[string]interface{}{
            "subnet_ids": terraform.Output(t, vpcOpts, "subnet_ids"), // where my subnet is the list received from the output of other module
        },

while this been implemented, i still face issue as when it passes the subnet_ids as the variable it is treated as a string of single unit, for ex. "[subnet-XXXX subnet-YYYY]" is a string and not a list while the output from the module was a list and not a string.

is there any way out to take the subnet value from the output as a list and not as string .

brikis98 commented 3 years ago

Most likely, Terratest is passing the value as a list, but you need to set the type on the subnet_ids input variable to list(string) so that Terraform knows to parse it as a list.

dimaunx commented 3 years ago

I'm am having similar issue as @rishabhanand26 ,

The destination terraform module variable is configured as list(string):

variable "private_subnets" {
  description = "Cluster private subnets"
  type        = list(string)
}

I pass the variable in a test stage as follows:

"private_subnets":  terraform.Output(t, awsNetworkOpts, "private_subnets")

Getting an error:

 Error Trace:   apply.go:15
                    aws_infra_test.go:98
                    test_structure.go:25
                    aws_infra_test.go:51
 Error:         Received unexpected error:
                 FatalError{Underlying: error while running command: exit status 1; 
                 Error: Missing item separator

                   on <value for var.private_subnets> line 1:
                   (source code not available)

                 Expected a comma to mark the beginning of the next item.

                 Error: No value for required variable

                   on variables.tf line 14:
                   14: variable "private_subnets" {

                 The root module input variable "private_subnets" is not set, and has no
                 default value. Use a -var or -var-file command line argument to provide a
                 value for this variable.
                 }

The actual output from the source module looks as follows: Outputs: private_subnets = [ "subnet-xxx", "subnet-xxx", ]

The var that is passed to the destination module is -var private_subnets=[subnet-xxx subnet-xxx] Seems like the output always is passed as string, if i pass a list it works.

For me the workaround was to use the https://github.com/Jeffail/gabs library.

awsNetworkOpts := test_structure.LoadTerraformOptions(t, awsNetworkModulePath)
jsonParsed, err := gabs.ParseJSON([]byte(terraform.OutputJson(t, awsNetworkOpts, "")))
if err != nil {
    t.Error(err)
}

obj, err := jsonParsed.JSONPointer("/private_subnets/value")
if err != nil {
    t.Error(err)
}

var privateSubnets []string
for _, child := range obj.Children() {
    privateSubnets = append(privateSubnets, child.Data().(string))
}

vars := map[string]interface{}{"private_subnets": privateSubnets}

To view the output in json format just fmt.Println(jsonParsed).

Hope it helps someone.

brikis98 commented 3 years ago

I pass the variable in a test stage as follows:

"private_subnets":  terraform.Output(t, awsNetworkOpts, "private_subnets")

That's your problem right there: terraform.Output returns a string. Use terraform.OutputList instead.

dimaunx commented 3 years ago

I pass the variable in a test stage as follows:

"private_subnets":  terraform.Output(t, awsNetworkOpts, "private_subnets")

That's your problem right there: terraform.Output returns a string. Use terraform.OutputList instead.

Thank you very much!