Support `terraform test` command

[suzuki-shunsuke]

Feature Overview

Support posting the result of terraform test command.

Why is the feature needed?

Terraform v1.6 or later supports terraform test command.

https://developer.hashicorp.com/terraform/language/tests

It would be nice if tfcmt can format the result human friendly and post the result.

Example Code

$ tfcmt test -- terraform test

Configuration

terraform:
  test:
    when_test_success:
      template: ""
    when_test_fail:
      template: ""
    when_parse_error:
      template: ""

note

No response

[suzuki-shunsuke]

terraform test has -json option. It would be much easier to parse JSON than to parse the raw output.

[suzuki-shunsuke]

The output of terraform test -json isn't what I expected. 😅

{"@level":"info","@message":"Terraform 1.7.4","@module":"terraform.ui","@timestamp":"2024-02-29T05:53:00.461999+09:00","terraform":"1.7.4","type":"version","ui":"1.2"}
{"@level":"info","@message":"Found 1 file and 3 run blocks","@module":"terraform.ui","@timestamp":"2024-02-29T05:53:00.468581+09:00","test_abstract":{"main.tftest.hcl":["check_bucket_name","check_id","check_again"]},"type":"test_abstract"}
{"@level":"info","@message":"main.tftest.hcl... in progress","@module":"terraform.ui","@testfile":"main.tftest.hcl","@timestamp":"2024-02-29T05:53:00.468616+09:00","test_file":{"path":"main.tftest.hcl","progress":"starting"},"type":"test_file"}
{"@level":"info","@message":"  \"check_bucket_name\"... in progress","@module":"terraform.ui","@testfile":"main.tftest.hcl","@testrun":"check_bucket_name","@timestamp":"2024-02-29T05:53:00.468629+09:00","test_run":{"path":"main.tftest.hcl","run":"check_bucket_name","progress":"starting","elapsed":0},"type":"test_run"}
{"@level":"info","@message":"  \"check_bucket_name\"... fail","@module":"terraform.ui","@testfile":"main.tftest.hcl","@testrun":"check_bucket_name","@timestamp":"2024-02-29T05:53:00.535913+09:00","test_run":{"path":"main.tftest.hcl","run":"check_bucket_name","progress":"complete","status":"fail"},"type":"test_run"}
{"@level":"error","@message":"Error: Test assertion failed","@module":"terraform.ui","@testfile":"main.tftest.hcl","@testrun":"check_bucket_name","@timestamp":"2024-02-29T05:53:00.536056+09:00","diagnostic":{"severity":"error","summary":"Test assertion failed","detail":"id is empty","range":{"filename":"main.tftest.hcl","start":{"line":3,"column":25,"byte":61},"end":{"line":3,"column":51,"byte":87}},"snippet":{"context":"run \"check_bucket_name\"","code":"        condition     = null_resource.foo.id == \"\"","start_line":3,"highlight_start_offset":24,"highlight_end_offset":50,"values":[{"traversal":"null_resource.foo.id","statement":"is \"8439948162146153121\""}]}},"type":"diagnostic"}
{"@level":"info","@message":"  \"check_id\"... in progress","@module":"terraform.ui","@testfile":"main.tftest.hcl","@testrun":"check_id","@timestamp":"2024-02-29T05:53:00.536113+09:00","test_run":{"path":"main.tftest.hcl","run":"check_id","progress":"starting","elapsed":0},"type":"test_run"}
{"@level":"info","@message":"  \"check_id\"... pass","@module":"terraform.ui","@testfile":"main.tftest.hcl","@testrun":"check_id","@timestamp":"2024-02-29T05:53:00.583278+09:00","test_run":{"path":"main.tftest.hcl","run":"check_id","progress":"complete","status":"pass"},"type":"test_run"}
{"@level":"info","@message":"  \"check_again\"... in progress","@module":"terraform.ui","@testfile":"main.tftest.hcl","@testrun":"check_again","@timestamp":"2024-02-29T05:53:00.583295+09:00","test_run":{"path":"main.tftest.hcl","run":"check_again","progress":"starting","elapsed":0},"type":"test_run"}
{"@level":"info","@message":"  \"check_again\"... fail","@module":"terraform.ui","@testfile":"main.tftest.hcl","@testrun":"check_again","@timestamp":"2024-02-29T05:53:00.631567+09:00","test_run":{"path":"main.tftest.hcl","run":"check_again","progress":"complete","status":"fail"},"type":"test_run"}
{"@level":"error","@message":"Error: Test assertion failed","@module":"terraform.ui","@testfile":"main.tftest.hcl","@testrun":"check_again","@timestamp":"2024-02-29T05:53:00.631701+09:00","diagnostic":{"severity":"error","summary":"Test assertion failed","detail":"id should not be empty","range":{"filename":"main.tftest.hcl","start":{"line":17,"column":25,"byte":312},"end":{"line":17,"column":51,"byte":338}},"snippet":{"context":"run \"check_again\"","code":"        condition     = null_resource.foo.id == \"\"","start_line":17,"highlight_start_offset":24,"highlight_end_offset":50,"values":[{"traversal":"null_resource.foo.id","statement":"is \"8439948162146153121\""}]}},"type":"diagnostic"}
{"@level":"info","@message":"main.tftest.hcl... tearing down","@module":"terraform.ui","@testfile":"main.tftest.hcl","@timestamp":"2024-02-29T05:53:00.631712+09:00","test_file":{"path":"main.tftest.hcl","progress":"teardown"},"type":"test_file"}
{"@level":"info","@message":"  \"check_again\"... tearing down","@module":"terraform.ui","@testfile":"main.tftest.hcl","@testrun":"check_again","@timestamp":"2024-02-29T05:53:00.631745+09:00","test_run":{"path":"main.tftest.hcl","run":"check_again","progress":"teardown","elapsed":0},"type":"test_run"}
{"@level":"info","@message":"main.tftest.hcl... fail","@module":"terraform.ui","@testfile":"main.tftest.hcl","@timestamp":"2024-02-29T05:53:00.681064+09:00","test_file":{"path":"main.tftest.hcl","progress":"complete","status":"fail"},"type":"test_file"}
{"@level":"info","@message":"Failure! 1 passed, 2 failed.","@module":"terraform.ui","@timestamp":"2024-02-29T05:53:00.681088+09:00","test_summary":{"status":"fail","passed":1,"failed":2,"errored":0,"skipped":0},"type":"test_summary"}

[suzuki-shunsuke]

Example output of terraform test

main.tftest.hcl... in progress
  run "check_bucket_name"... fail
╷
│ Error: Test assertion failed
│ 
│   on main.tftest.hcl line 3, in run "check_bucket_name":
│    3:         condition     = null_resource.foo.id == ""
│     ├────────────────
│     │ null_resource.foo.id is "3981582370779221075"
│ 
│ id is empty
╵
  run "check_id"... pass
  run "check_again"... fail
╷
│ Error: Test assertion failed
│ 
│   on main.tftest.hcl line 17, in run "check_again":
│   17:         condition     = null_resource.foo.id == ""
│     ├────────────────
│     │ null_resource.foo.id is "3981582370779221075"
│ 
│ id should not be empty
╵
main.tftest.hcl... tearing down
main.tftest.hcl... fail

[suzuki-shunsuke]

Draft

:x: Test Failed

Failure! 0 passed, 1 failed.

• main.tftest.hcl check_bucket_name: id is empty

Test Result (Click me)

```hcl main.tftest.hcl... in progress run "check_bucket_name"... fail ╷ │ Error: Test assertion failed │ │ on main.tftest.hcl line 3, in run "check_bucket_name": │ 3: condition = null_resource.foo.id == "" │ ├──────────────── │ │ null_resource.foo.id is "5238291740601994744" │ │ id is empty ╵ main.tftest.hcl... tearing down main.tftest.hcl... fail Failure! 0 passed, 1 failed. ```