Tests feel brittle if I cannot specify an example input file with each.

[cep21]

Hi!

Thanks so much for the wonderful tool. I'm trying to write unit tests at the moment for a terraform check. I can run conftest parse to see what the input is, then copy/paste that into a test file, but that seems brittle since I'm never re-checking that the test input matches the file's parse and I'm trusting that parse won't change or add fields and that I copied it correctly.

Ideally, I would have a way to specify the entire "test" terraform file and throw it at the test. One example may be

deny_s3[msg] {
        s3_storage_key := input.terraform.backend.s3
        msg = "We don't allow s3 backends anymore"
}

Then the test

test1 = "
terraform {
  backend "aws" {
  }
}
"
test2 = "
terraform {
  backend "localfile" {
  }
}
"
test3 = "
terraform {
  backend "localfile" {
  }
}
"
test_allows {
        deny_s3 with input as parse(test1)
        deny_s3 with input as parse(test2)
}
test_allow {
        empty(deny) with input as parse(test3)
}

Or maybe test1, test2, and test3 are filenames I reference.

What I'm doing right now is I have a directory /policy/testdata/mustpass and /policy/testdata/mustfail, with a bunch of terraform files in them and I just run conftest against both directories expecting a pass or fail for each. Maybe this is the recommended way to test with conftest? It's strange since I just skip the conftest verify subcommand entirely. Ideas or thoughts welcome!

[jalseth]

Hi @cep21, thanks for opening the issue. To clarify, you'd like to have Terraform (or other) formatted source files for the unit tests (conftest verify) to run against, correct? If so, I agree this could be a useful feature to implement to enable syntax checks, etc. in the test resource files. If you have time, I'd welcome a PR, otherwise I can look to add this when I have time.

[cep21]

That's right. I probably won't be able to implement this myself since I have a few workarounds (the directory mentioned above, and the easier workaround of not writing tests :) ) I'm very open on how this works and am curious what you think the best way is. A few may be

1. Embed the terraform file as a text inside the test itself, with a parse function of some kind.
2. Have a directory of files that are pre-seeded as data to the test. Unsure what this data would look like. Maybe testdata.example-1.X ?

[jalseth]

@cep21 I've implemented the first option in #726 for inline parsing of configurations using a new parse_config() function. This will be included in the next conftest release.

[jalseth]

parse_config_file() was also included in that PR, which handles case #2. Going to close this issue.

This will be in the next release of conftest, likely ~1 week from now after OPA's next feature release.