Use deterministic-zip instead of built-in zip command

[glenthomas]

Is your request related to a new offering from AWS?

N/A

Is your request related to a problem? Please describe.

I want this module to build the lambda deployment package, but I also need the source code hash to be consistent to avoid problems when running a diff on Terraform plans for the same source code. The built-in zip functionality for this module produces inconsistent hashes (or archive filename if that's not the same thing..?).

Describe the solution you'd like.

I would like to use the same zip technique as exists in timo-reymann/deterministic-zip. This utility will create uniform zip packages by overwriting file metadata like the last modified date time and adding files to the archive in a consistent order.

Describe alternatives you've considered.

My module configuration to build the package looks like this, but source code hashes are inconsistent:

module "metrics_filter_lambda_function" {
  source  = "terraform-aws-modules/lambda/aws"
  version = "6.4.0"

  source_path = [
    {
      path = "${path.module}/metrics_lambda",
      commands = [
        ":zip",                          # required for source code hashing
        "npm test",                      # run tests
        "npx tsc",                       # transpile Typescript code
        "npm install --production=true", # remove dev dependencies
        ":zip"                           # create deployment package
      ],
      patterns = [
        "!.*",             # exclude all files by default
        "dist/.+",         # include transpiled javascript files
        "node_modules/.+", # include node modules
      ],
      npm_requirements = true
    }
  ]
}

I would like to do something like this, but not sure how

module "metrics_filter_lambda_function" {
  source  = "terraform-aws-modules/lambda/aws"
  version = "6.4.0"

  source_path = [
    {
      path = "${path.module}/metrics_lambda",
      commands = [
        "deterministic-zip",             # required for source code hashing
        "npm test",                      # run tests
        "npx tsc",                       # transpile Typescript code
        "npm install --production=true", # remove dev dependencies
        "deterministic-zip"              # create deployment package
      ],
      patterns = [
        "!.*",             # exclude all files by default
        "dist/.+",         # include transpiled javascript files
        "node_modules/.+", # include node modules
      ],
      npm_requirements = true
    }
  ]
}

This results in this error:

Planning failed. Terraform encountered an error while generating this plan. │ Error: External Program Execution Failed │ │ with module.cloudtrail.module.metrics_filter_lambda_function.data.external.archive_prepare[0], │ on .terraform/modules/cloudtrail.metrics_filter_lambda_function/package.tf line 10, in data "external" "archive_prepare": │ 10: program = [local.python, "${path.module}/package.py", "prepare"] │ │ The data source received an unexpected error while attempting to execute the program. │ │ Program: /opt/homebrew/bin/python3 │ Error Message: Traceback (most recent call last): │ File "/Users/glen.thomas/repos/aws-security/infra/.terraform/modules/cloudtrail.metrics_filter_lambda_function/package.py", line 1632, in │ main() │ File "/Users/glen.thomas/repos/aws-security/infra/.terraform/modules/cloudtrail.metrics_filter_lambda_function/package.py", line 1628, in main │ exit(args.command(args)) │ ^^^^^^^^^^^^^^^^^^ │ File "/Users/glen.thomas/repos/aws-security/infra/.terraform/modules/cloudtrail.metrics_filter_lambda_function/package.py", line 1435, in prepare_command │ content_hash = bpm.hash(hash_extra_paths) │ ^^^^^^^^^^^^^^^^^^^^^^^^^^ │ File "/Users/glen.thomas/repos/aws-security/infra/.terraform/modules/cloudtrail.metrics_filter_lambda_function/package.py", line 641, in hash │ raise ValueError('BuildPlanManager.plan() should be called first') │ ValueError: BuildPlanManager.plan() should be called first │ │ State: exit status 1

I have tried using :zip for the first command and deterministic-zip for the last:

module "metrics_filter_lambda_function" {
  source  = "terraform-aws-modules/lambda/aws"
  version = "6.4.0"

  source_path = [
    {
      path = "${path.module}/metrics_lambda",
      commands = [
        ":zip",                          # required for source code hashing
        "npm test",                      # run tests
        "npx tsc",                       # transpile Typescript code
        "npm install --production=true", # remove dev dependencies
        "deterministic-zip"              # create deployment package
      ],
      patterns = [
        "!.*",             # exclude all files by default
        "dist/.+",         # include transpiled javascript files
        "node_modules/.+", # include node modules
      ],
      npm_requirements = true
    }
  ]
}

This results in this error:

╷ │ Error: Provider produced inconsistent final plan │ │ When expanding the plan for module.cloudtrail.module.metrics_filter_lambda_function.aws_lambda_function.this[0] to include > new values learned so │ far during apply, provider "registry.terraform.io/hashicorp/aws" produced an invalid new value for .source_code_hash: was │ cty.StringVal("WSFxt+6d0x6ikfQcoy8PYrtNJf8F7kULoSO1MHvgDPk="), but now cty.StringVal("K3kK+A/uFV8QMIp9m2Ilw07ugfitMg/w+7KBASdsVN0="). │ │ This is a bug in the provider, which should be reported in the provider's own issue tracker. ╵

Additional context

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open 30 days with no activity. Remove stale label or comment or this issue will be closed in 10 days

[github-actions[bot]]

This issue was automatically closed because of stale in 10 days

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.