[Bug]: `aws_lambda_layer_version` stores wrong value of `source_code_hash` to state

[AlexStasko]

Terraform Core Version

1.3.6

AWS Provider Version

4.39.0

Affected Resource(s)

• aws_lambda_layer_version

Expected Behavior

The value passed to source_code_hash should be correctly stored in tf state.

Actual Behavior

I'm passing a hash value of the file calculated by the external data source to two resources, null_resource, which downloads file that should be stored in Layer, and aws_lambda_layer_version, which creates Layer. After first terraform apply, I see that the hashes in these two resources are different. When I run terraform plan a second time, tf calculates that source_code_hash was changed and set the correct value. But in this case, null_resource will not be triggered as the hash of the file didn't change, and terraform apply fails with an error that the file doesn't exist.

Relevant Error/Panic Output Snippet

No response

Terraform Configuration Files

cdk.tf.json:

{
  "//": {
    "metadata": {
      "backend": "local",
      "stackName": "test",
      "version": "0.14.3"
    }
  },
  "data": {
    "external": {
      "File-eu-west-1": {
        "//": {
          "metadata": {
            "path": "test/File-eu-west-1",
            "uniqueId": "File-eu-west-1"
          }
        },
        "program": [
          "/bin/sh",
          "/home/user/calculate-hash.sh"
        ],
        "query": {
          "downloadUrl": "${var.url}",
          "fileName": "archive.zip"
        }
      }
    }
  },
  "provider": {
    "aws": [
      {
        "alias": "eu-west-1",
        "region": "eu-west-1"
      }
    ],
    "external": [
      {
      }
    ],
    "null": [
      {
      }
    ]
  },
  "resource": {
    "aws_lambda_layer_version": {
      "LambdaLayer-eu-west-1": {
        "//": {
          "metadata": {
            "path": "test/LambdaLayer-eu-west-1",
            "uniqueId": "LambdaLayer-eu-west-1"
          }
        },
        "compatible_architectures": [
          "x86_64",
          "arm64"
        ],
        "compatible_runtimes": [
          "nodejs14.x",
          "nodejs16.x"
        ],
        "depends_on": [
          "data.external.File-eu-west-1",
          "null_resource.LocalFile-eu-west-1"
        ],
        "filename": "bundle-eu-west-1.zip",
        "layer_name": "test-layer",
        "provider": "aws.eu-west-1",
        "skip_destroy": true,
        "source_code_hash": "${data.external.File-eu-west-1.result[\"hash\"]}"
      }
    },
    "null_resource": {
      "LocalFile-eu-west-1": {
        "//": {
          "metadata": {
            "path": "test/LocalFile-eu-west-1",
            "uniqueId": "LocalFile-eu-west-1"
          }
        },
        "depends_on": [
          "data.external.File-eu-west-1"
        ],
        "provisioner": [
          {
            "local-exec": {
              "command": "curl -s -o archive.zip '${var.url}' && zip bundle-eu-west-1.zip archive.zip"
            }
          }
        ],
        "triggers": {
          "hash": "${data.external.File-eu-west-1.result[\"hash\"]}"
        }
      }
    }
  },
  "terraform": {
    "backend": {
      "local": {
        "path": "/home/user/terraform.astra.tfstate"
      }
    },
    "required_providers": {
      "aws": {
        "source": "aws",
        "version": "4.39.0"
      },
      "external": {
        "source": "external",
        "version": "2.2.3"
      },
      "null": {
        "source": "null",
        "version": "3.2.1"
      }
    }
  },
  "variable": {
    "url": {
      "description": "Url to download file",
      "sensitive": false,
      "type": "string"
    }
  }
}

Steps to Reproduce

• Apply terraform configuration
• Check state file (hashes should be different)
• Run terraform plan (layer resource should have changed hash)

Debug Output

No response

Panic Output

No response

Important Factoids

I'm using CDKTF to generate terraform configuration

References

No response

Would you like to implement a fix?

None

[github-actions[bot]]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[justinretzolk]

Hey @AlexStasko 👋 Thank you for taking the time to raise this! There's a bit about your configuration (particularly around what the hash calculation script looks like) that I'm making some assumptions about, but I think I might have some ideas as to what's causing this. This feels a bit like a Terraform Core issue based on my assumptions, but I'd like to verify my thoughts before we discuss moving the issue over to that repository.

The external data source has no dependency on the null_resource data source; exactly the opposite, in fact -- the null_resource is dependent upon the external resource. The external resource, however, is calculating a hash value for a file that (as far as I can tell) is being downloaded by the null_resource. It would seem based on this, that the script in the external data source might be returning a hash value that's not what you're expecting it to be.

I do feel like I might be missing a key piece of information here that would make this a bit more clear to me. If possible, can you supply debug logs (redacted as needed) so that we can review those as well?

[justinretzolk]

Since we haven't heard back in a while, I'm going to close this issue out. If you're still having troubles, or have future issues with the AWS Provider, please feel free to open a new issue (referencing this one for context as needed).

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.