`handler and runtime must be set when PackageType is Zip` when `handler` is unknown

[lukehoban]

Running pulumi preview on this program:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const bucket = new aws.s3.BucketV2("MyBucket", {});

const lambdaRole = new aws.iam.Role("lambdaRole", {
  assumeRolePolicy: aws.iam.assumeRolePolicyForPrincipal({Service: "lambda.amazonaws.com"}),
});

new aws.iam.RolePolicyAttachment("lambdaRolePolicyAttachment", {
  role: lambdaRole.name,
  policyArn: aws.iam.ManagedPolicy.AWSLambdaBasicExecutionRole,
});

const handler = bucket.arn.apply(bucketArn => {
  return "lambda.handler";
});

const fn = new aws.lambda.Function("MyFunction", {
  role: lambdaRole.arn,
  code: new pulumi.asset.AssetArchive({
    "lambda.mjs": new pulumi.asset.FileAsset("./lambda.mjs"),
  }),
  runtime: "nodejs20.x",
  handler,
});

Leads to:

Previewing update (dev)

View in Browser (Ctrl+O): https://app.pulumi.com/upstarts/bucketlambdaerror/dev/previews/5daf9adc-f41a-4c18-90f2-a66fea2f2a3a

     Type                             Name                        Plan       Info
 +   pulumi:pulumi:Stack              bucketlambdaerror-dev       create     1 error
 +   ├─ aws:s3:BucketV2               MyBucket                    create     
 +   ├─ aws:iam:Role                  lambdaRole                  create     
 +   ├─ aws:iam:RolePolicyAttachment  lambdaRolePolicyAttachment  create     
 +   └─ aws:lambda:Function           MyFunction                  create     2 errors

Diagnostics:
  pulumi:pulumi:Stack (bucketlambdaerror-dev):
    error: preview failed

  aws:lambda:Function (MyFunction):
    error:   sdk-v2/provider2.go:509: sdk.helper_schema: handler and runtime must be set when PackageType is Zip: provider=aws@6.54.2
    error: Preview failed: diffing urn:pulumi:dev::bucketlambdaerror::aws:lambda/function:Function::MyFunction: 1 error occurred:
        * handler and runtime must be set when PackageType is Zip

There are at least two problems here:

1. The diff fails
2. An esoteric Go error message is printed to the user even though they are a TypeScript user

The gRPC debugging on the provider shows this:

{
    "method": "/pulumirpc.ResourceProvider/Create",
    "request": {
        "urn": "urn:pulumi:dev::bucketlambdaerror::aws:lambda/function:Function::MyFunction",
        "properties": {
            "__defaults": [
                "memorySize",
                "name",
                "packageType",
                "publish",
                "reservedConcurrentExecutions",
                "skipDestroy",
                "timeout"
            ],
            "code": {
                "4dabf18193072939515e22adb298388d": "0def7320c3a5731c473e5ecbe6d01bc7",
                "assets": {
                    "lambda.mjs": {
                        "4dabf18193072939515e22adb298388d": "c44067f5952c0a294b673a41bacd8c17",
                        "hash": "4a4d08e9b7385019c1d7f2516656ef082a8ddaefa3ec5efcbc12144bb8e5ea76",
                        "path": "./lambda.mjs"
                    }
                },
                "hash": "ea838fca8fbcd362c3a387bd379a250ec41e4cd2580d4ef9e6f06784b32bde56"
            },
            "handler": "04da6b54-80e4-46f7-96ec-b56ff0331ba9",
            "memorySize": 128,
            "name": "MyFunction-e3cc8c7",
            "packageType": "Zip",
            "publish": false,
            "reservedConcurrentExecutions": -1,
            "role": "04da6b54-80e4-46f7-96ec-b56ff0331ba9",
            "runtime": "nodejs20.x",
            "skipDestroy": false,
            "timeout": 3
        },
        "preview": true,
        "name": "MyFunction",
        "type": "aws:lambda/function:Function"
    },
    "errors": [
        "rpc error: code = Unknown desc = diffing urn:pulumi:dev::bucketlambdaerror::aws:lambda/function:Function::MyFunction: 1 error occurred:\n\t* handler and runtime must be set when PackageType is Zip\n\n"
    ],
    "metadata": {
        "kind": "resource",
        "mode": "client",
        "name": "aws"
    }
}

Notably, as expected, the handler value is the unknown placeholder value 04da6b54-80e4-46f7-96ec-b56ff0331ba9. But this then leads to a failure instead of being properly converted into an unknown for the diff on the TF provider side.

[t0yv0]

Got the same issue in Terraform when an unknown is passed to handler. We will need to file an upstream issue. Suspect missing/unknown distinction in the internals of SDKv2 TF framework is making some upstream code error-prone here.

data "aws_iam_policy_document" "assume_role" {
  statement {
    effect = "Allow"

    principals {
      type        = "Service"
      identifiers = ["lambda.amazonaws.com"]
    }

    actions = ["sts:AssumeRole"]
  }
}

resource "aws_iam_role" "iam_for_lambda" {
  name               = "iam_for_lambda"
  assume_role_policy = data.aws_iam_policy_document.assume_role.json
}

data "archive_file" "lambda" {
  type        = "zip"
  source_file = "lambda.js"
  output_path = "lambda_function_payload.zip"
}

resource "aws_lambda_function" "test_lambda" {
  # If the file is not in the current working directory you will need to include a
  # path.module in the filename.
  filename      = "lambda_function_payload.zip"
  function_name = "lambda_function_name"
  role          = aws_iam_role.iam_for_lambda.arn
  handler       = aws_iam_role.iam_for_lambda.arn # force this to unknown

  source_code_hash = data.archive_file.lambda.output_base64sha256

  runtime = "nodejs18.x"

  environment {
    variables = {
      foo = "bar"
    }
  }
}

Got this plan:

Plan: 1 to add, 0 to change, 0 to destroy.
╷
│ Error: handler and runtime must be set when PackageType is Zip
│ 
│   with aws_lambda_function.test_lambda,
│   on infra.tf line 25, in resource "aws_lambda_function" "test_lambda":
│   25: resource "aws_lambda_function" "test_lambda" {
│ 
╵

[VenelinMartinov]

This is happening here: https://github.com/hashicorp/terraform-provider-aws/blob/94d3e1a0b79324b36f50befceac79c122c82018a/internal/service/lambda/function.go#L1325

DiffCustomizers can not distinguish between unspecified and unknown as they get the plain go values - we had the same issue in GCP:

https://github.com/pulumi/pulumi-gcp/issues/2372 fixes: https://github.com/pulumi/pulumi-terraform-bridge/pull/2417 and https://github.com/pulumi/pulumi-gcp/pull/2386

Note that he issue in GCP was the other way around - empty values were being treated as unknown instead.

[t0yv0]

Luke noted NewValueKnown() bool API in https://github.com/hashicorp/terraform-plugin-sdk/blob/dd6c1007928eb4058ce836752fd9ec06d304941e/helper/schema/resource_diff.go#L423 that might possibly be part of fixing upstream here.