gruntwork-io / terratest

Terratest is a Go library that makes it easier to write automated tests for your infrastructure code.
https://terratest.gruntwork.io/
Apache License 2.0
7.51k stars 1.33k forks source link

I want to generate a html report for terratest scripts. Like how many tests are Pass/Fail? If it is possible can you tell me how to do it? #939

Open tmitra1990 opened 3 years ago

yorinasub17 commented 3 years ago

terratest_log_parser generates a junit xml report which many CI systems will automatically render for you.

If you don't have anything handy, you can use something like https://gitlab.com/inorton/junit2html

jghal commented 2 years ago

Any hints on how to write our test case(s) file(s) to get more useful junit reports? I only get something like this

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
    <testsuite tests="1" failures="0" time="309.730" name="command-line-arguments">
        <properties>
            <property name="go.version" value="go1.13"></property>
        </properties>
        <testcase classname="command-line-arguments" name="TestAws" time="309.560"></testcase>
    </testsuite>
</testsuites>

Right now I have one source file for the test cases, and it defines one function like

func TestAws(t *testing.T) {
    terraformOptions := &terraform.Options{
        TerraformDir: "..",
        Vars: map[string]interface{}{
            ...
        },
    }
    defer terraform.Destroy(t, terraformOptions)
    terraform.InitAndApply(t, terraformOptions)
    aws.AssertS3BucketExists(t, awsRegion, "expected_bucket_name")
    dynamoTable := aws.GetDynamoDBTable(t, awsRegion, "expected_table_name")
    assert.NotNil(t, dynamoTable)
}

I can infer that each func which takes the t *testing.T parameter translates to a testcase in the report. But I don't want to apply and destroy the module I'm testing multiple times. I'm assuming I need to make use of test_structure here? Because if so, looking at examples terraform_gcp_ig_example_test.go you are indeed doing a fully apply and destroy.

jkrusic commented 2 months ago

Hello,

We are adding terratest UnitTests for our Terraform code as-well, however the JUNIT report is reporting 0 testsuite cases, even though when running locally we have multiple reports. The information isn't very useful, is there a work around?

I have multiple t.Run() statements, so I am expecting the number of testsuite tests to match the number of Test groupings.

<testsuites>
<testsuite tests="0" failures="0" time="144.055" name="tests">
<properties>
<property name="go.version" value="go1.21.1"/>
</properties>
</testsuite>
</testsuites>
iron-rain commented 1 month ago

I hit the same issue as @jkrusic, my issue was not using the verbose flag on the test command:

go test -timeout 30m | tee test_output.log

when it needed to be:

go test -timeout 30m -v | tee test_output.log

Not sure if it's a bug as I'm new to Terratest, or if the docs just need an update (https://terratest.gruntwork.io/docs/testing-best-practices/debugging-interleaved-test-output/), but the first option doesn't include the markings (like below) that get used by the log_parser:

=== RUN   TestKubernetesHelloWorldExample
=== PAUSE TestKubernetesHelloWorldExample
=== CONT  TestKubernetesHelloWorldExample

--- PASS: TestKubernetesHelloWorldExample (1.88s)
jkrusic commented 1 month ago

@iron-rain ,

Thank you, that fixed my issues. Now getting what I expected. Appreciate the comment.