Render JSON String Diffs as JSON

[jcjones]

Proposal

In Terraform plan and apply diff output, when a string evaluated looks like JSON, the renderer calls a function renderStringDiffAsJson which pretty-prints the output.

It would be lovely to have available to Nomad the same capability for understanding diffs of pre-rendered JSON blocks, for when rendering via jsonencode is impractical. This would serve to reduce human errors when evaluating the correctness of plan outputs.

Use-cases

Job specifications which render out JSON can be difficult to evaluate in a plan. Take for example this change, which changes a setting maxConnectionAge from 60s to 30s in an inline JSON template:

     +/- Job: "boulder-nonce"
     +/- Task Group: "nonce" (1 canary, 1 ignore)
       +/- Task: "boulder-nonce" (forces create/destroy update)
         +/- Template {
               ChangeMode:    "restart"
               ChangeSignal:  ""
               DestPath:      "${NOMAD_TASK_DIR}/nonce.json"
           +/- EmbeddedTmpl:  "{\n  \"NonceService\": {\n    \"maxUsed\": {{ key \"service/nonce/maxUsed\" | parseInt }},\n    \"noncePrefix\": \"{{ slice (env \"NOMAD_ADDR_grpc\" | sha256Hex) 0 4 | toUpper }}\",\n    \"debugAddr\": \"{{ env \"NOMAD_ADDR_debug\" }}\",\n    \"grpc\": {\n      \"address\": \"{{ env \"NOMAD_ADDR_grpc\" }}\",\n      \"maxConnectionAge\": \"60s\",\n      \"services\": {\n        \"nonce.NonceService\": {\n          \"clientNames\": {{ key \"service/nonce/clientNames\" }}\n        },\n        \"grpc.health.v1.Health\": {\n          \"clientNames\": [\n            \"ifn-worker.{{ env \"meta.le_env_service\" }}\"\n          ]\n        }\n      }\n    },\n    \"tls\": {\n      \"caCertFile\": \"secrets/nonce/ca-cert.pem\",\n      \"certFile\": \"secrets/nonce/cert.pem\",\n      \"keyFile\": \"secrets/nonce/key.pem\"\n    },\n    \"syslog\": {{ keyOrDefault \"service/nonce/syslog\" \"{}\" }}\n  }\n}\n" => "{\n  \"NonceService\": {\n    \"maxUsed\": {{ key \"service/nonce/maxUsed\" | parseInt }},\n    \"noncePrefix\": \"{{ slice (env \"NOMAD_ADDR_grpc\" | sha256Hex) 0 4 | toUpper }}\",\n    \"debugAddr\": \"{{ env \"NOMAD_ADDR_debug\" }}\",\n    \"grpc\": {\n      \"address\": \"{{ env \"NOMAD_ADDR_grpc\" }}\",\n      \"maxConnectionAge\": \"30s\",\n      \"services\": {\n        \"nonce.NonceService\": {\n          \"clientNames\": {{ key \"service/nonce/clientNames\" }}\n        },\n        \"grpc.health.v1.Health\": {\n          \"clientNames\": [\n            \"ifn-worker.{{ env \"meta.le_env_service\" }}\"\n          ]\n        }\n      }\n    },\n    \"tls\": {\n      \"caCertFile\": \"secrets/nonce/ca-cert.pem\",\n      \"certFile\": \"secrets/nonce/cert.pem\",\n      \"keyFile\": \"secrets/nonce/key.pem\"\n    },\n    \"syslog\": {{ keyOrDefault \"service/nonce/syslog\" \"{}\" }}\n  }\n}\n"
               Envvars:       "false"
               ErrMissingKey: "false"
               LeftDelim:     "{{"
               Perms:         "0644"
               RightDelim:    "}}"
               SourcePath:    ""
               Splay:         "30000000000"
               VaultGrace:    "0"
             }
           Task: "generate-shutdown-helper"

Attempted Solutions

It's possible to externally decode the A => B syntax into a diff, though complicated by embedded consul template statements. By comparison, Terraform's rendering makes such a diff immediately human-readable.

An optimal solution is to not inline JSON, but rather use native HCL structures and the jsonencode tool. However, that can be difficult when still relying on runtime lookups or consul templates. So for codebases that are still migrating, JSON rendering would reduce human errors while validating plan output.

[tgross]

I had no idea Terraform did that, very clever. :grinning: I'll get this into roadmapping, but it also looks like the sort of thing that someone from the community might want to try to pick up so I'll mark it appropriately.

[pglushko]

Hi @tgross , Is issue still valid? If yes, I would like to contribute

Thanks

[tgross]

Yes, @PavloGl! You'll find the area of code of interest to be in nomad/structs/diff.go (with tests in diff_test.go that you'll want to add to). Thanks!

[pglushko]

Thanks for heads up!