Terraform issue - Unable to parse json

jbstans commented 2 weeks ago

Describe the issue

If you have a section of terraform like this:

data "local_file" "requests_layer" {

  filename = "../../deployment_artifacts/requests_py.zip"

}

In the Terraform Plan it uses the output of the plan and transforms it into JSON using the terraform show command

in the resulting terraform.json file there is a section like this.... notice the"content" section

"resources": [

    {

        "address": "module.fdd_common.data.local_file.requests_layer",

        "mode": "data",

        "type": "local_file",

        "name": "requests_layer",

        "provider_name": "registry.terraform.io/hashicorp/local",

        "schema_version": 0,

        "values": {

            "content": "PK\u0003\u0004\u0014\u0000\u0008\u0000\u0008\u0000\u0000\u0000aM\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0015\u0000\t\u0000python/bin/normalizerUT\u0005\u0000\u0001\u0000B�[\u003c��J�0\u0014@��\u0015W\u0026��5Q+\"��/��䒥���6\tɝ�������p��͍�I�\u0018�P/1N�����*c\u000c�5\u000f������=p��϶j\u0003�]\u0003\u003e�\u001c�;8�м���9�,��Be-j�q\u0006?�\\H0�\u003c���)\u001b?1�\u0007~b�Iȋ�\u0001\u0010��\t\u0011�\u000e4��8 ..........",

            "content_base64": "UEsDBBQACAAIAAA................ etc etc etc"

            "content_base64sha256": "d6zP5uMqvXSbJToSSLkcgoc+BibkaoStt7HgrDJ3PuI=",

            "content_base64sha512": "bWpoUOxoKNeux+w4nWtahbRsLa6Q/2lKdwUcUKXrikdMSwEC7IjOpUL5B5aFNE4tbTK4Z7h3PhL2xHQ7TVRpAw==",

            "content_md5": "a0993a9d11190812952aa66ef12cddab",

            "content_sha1": "aa61e1e57bb5be809c6dba8535bb7bcaea59475e",

            "content_sha256": "77accfe6e32abd749b253a1248b91c82873e0626e46a84adb7b1e0ac32773ee2",

            "content_sha512": "6d6a6850ec6828d7aec7ec389d6b5a85b46c2dae90ff694a77051c50a5eb8a474c4b0102ec88cea542f9079685344e2d6d32b867b8773e12f6c4743b4d546903",

            "filename": "../../deployment_artifacts/requests_py.zip",

            "id": "aa61e1e57bb5be809c6dba8535bb7bcaea59475e"

        },

        "sensitive_values": {}

    },

Checkov then fails with:

Error parsing file /path/to/the.json

itariq20 commented 2 weeks ago

@jbstans Hi, I was able to replicate this issue by taking a .tf file, and just changing the extension to .tf.json and certain files give that error upon being scanned. So potentially theres some format issue with that file, it's possible that the file is not exactly formatted correctly for the .tf.json format and hence we see that error.

jbstans commented 1 week ago

Interesting. In this instance the json is valid. My best guess from looking at the file was that it was because terraform loads the binary from the zip into the json. And the unicode from that breaks it.

I'm not sure if there's much you can do. We've been able to work around it, but I thought it worth flagging as you might be able to handle or ignore the unicode.

Saarett commented 1 week ago

Thanks @jbstans . I'll close the issue for now. If by any chance anyone else encounters this issue in the future, please reference to this thread.

bridgecrewio / checkov

Terraform issue - Unable to parse json #6446