Lambda Provisioned Concurrency Cannot Change Simultaneously with Alias

[ghost]

This issue was originally opened by @TheFLHurricane as hashicorp/terraform#24957. It was migrated here as a result of the provider split. The original body of the issue is below.

---

I have encountered an issue when using aws_lambda_provisioned_concurrency_config with a lambda alias. In most scenarios, this will work fine. However if I am simultaneously updating the Lambda (causing a new version to be published and the alias to be moved to that new version) while also updating the provisioned_concurrent_executions, I will fail with the message "InvalidParameterValueException: Alias with weights can not be used with Provisioned Concurrency"

It seems the issue is that the alias is rolling out a canary deployment, which causes the alias to be weighted for a period of time and prevents terraform from then updating the provisioned concurrency immediately afterward.

An ideal solution would be to have provisioned concurrency play nice by allowing the current provisioned version to spin down and spinning back up the correct number with the new version (in fact it seems reasonable to recreate this resource anytime the qualifying version/alias changes) Also reasonable would be adding an AllAtOnce Deployment Preference option to the alias to bypass the canary roll out.

Terraform Version

Terraform v0.12.24

Terraform Configuration Files

resource "aws_lambda_alias" "alias" {
  name             = "alias"
  description      = "alias"
  function_name    = aws_lambda_function.lambda.function_name
  function_version = aws_lambda_function.lambda.version
}

resource "aws_lambda_provisioned_concurrency_config" "lambda" {
  function_name                     = aws_lambda_function.lambda.function_name
  provisioned_concurrent_executions = 2
  qualifier                         = aws_lambda_alias.alias.name
}

Debug Output

Crash Output

(When changing both alias function_version and provisioned concurrency provisioned_concurrent_executions) On Apply:

Error: error putting Lambda Provisioned Concurrency Config (lambda-function:alias): InvalidParameterValueException: Alias with weights can not be used with Provisioned Concurrency { RespMetadata: { StatusCode: 400, RequestID: "82b1c399-056e-41b7-96e5-821593e992f6" }, Message_: "Alias with weights can not be used with Provisioned Concurrency", Type: "User" }

Expected Behavior

Provisioned concurrency with new version should succeed

Actual Behavior

Provisioned concurrency fails with InvalidParameterValueException

Steps to Reproduce

1. Run the above resources with any aws_lambda_function.lambda resource
2. Deploy a new version of the lambda and change provisioned_concurrent_executions and run again.

Additional Context

References

[MrDionysus]

I've used this as a successful hack/workaround after experiencing the same problem. Would love to see a cleaner solution! The delay time of 240 worked well for 10 provisioned executions.

#Provisioned Concurrency
resource "aws_lambda_alias" "function_alias" {
  name             = "authorizeralias"
  function_name    = "${aws_lambda_function.authorizer_lambda.arn}"
  function_version = "${aws_lambda_function.authorizer_lambda.version}"
 }

resource "null_resource" "alias_provisioned_concurrency_transition_delay" {
  depends_on = ["aws_lambda_alias.function_alias"]
  provisioner "local-exec" {
   command = "sleep 240"
  }
  triggers = {
     function_version = "${aws_lambda_function.authorizer_lambda.version}"
  }
}

resource "aws_lambda_provisioned_concurrency_config" "function" {
  depends_on = ["null_resource.alias_provisioned_concurrency_transition_delay"]
  function_name                     = "${aws_lambda_function.authorizer_lambda.function_name}"
  provisioned_concurrent_executions = "${var.provisioned_executions}"
  qualifier                         = "${aws_lambda_alias.function_alias.name}"
}

[Clete2]

I can confirm @MrDionysus that your fix works for me. I will use it until a better alternative can be found.

[cema-sp]

Experiencing similar issues and was able to fix by adding an empty routing configuration to the alias

routing_config {}

[saidsef]

+1 on @MrDionysus solution, would be better if TF actually waited until the resource is ready.

[justinretzolk]

Hi @TheFLHurricane 👋 Thank you for taking the time to submit this issue. Given that there's been a few Terraform and AWS Provider releases since you initially reported it, can you confirm whether you're still running into this?

[hav]

I can confirm that this issue is still happening with terraform-aws-provider 3.25 and Terraform 0.13 at least.

[justinretzolk]

Hey @hav 👋 Thank you for the update. With that in mind, I'm going to go ahead and label this as a bug so that the team may look into it as time allows.

[YoniMelki]

@justinretzolk, Any update here 😊 ?

[justinretzolk]

Hey @YoniMelki 👋 Thank you for checking in on this. Unfortunately, I'm not able to provide an estimate on when this will be looked into due to the potential of shifting priorities (we prioritize work by count of ":+1:" reactions, as well as a few other things). For more information on how we prioritize, check out out prioritization guide.

[jnovick]

I am facing the same issue, but the work around did not work for me. Any advice? I tried what MrDionysus suggested and what cema-sp suggested. Neither worked.

[tgorton617]

I ran into this and didn't want to rely on a fixed timeout, so instead I used a check-and-wait loop using the AWS CLI inside a null resource:

resource "null_resource" "wait_for_alias_not_have_weights" {
  triggers = {
    always_run = timestamp()
  }

  depends_on = [aws_lambda_alias.api_current]

  provisioner "local-exec" {
    # loop until the alias is not weighted
    command = <<EOF
    while : 
    do
      echo 'Checking for weights on API lambda alias...'
      weights=$(aws lambda get-alias --function-name ${aws_lambda_function.api.function_name} --name ${local.api_alias_name} --query 'RoutingConfig' || echo null)
      if [ "$weights" == "null" ]; then
        break
      fi
      sleep 5
    done
    EOF
  }
}

resource "aws_lambda_alias" "api_current" {
  name             = local.api_alias_name
  function_name    = aws_lambda_function.api.arn
  function_version = aws_lambda_function.api.version
}

resource "aws_lambda_provisioned_concurrency_config" "api" {
  depends_on = [null_resource.wait_for_alias_not_have_weights]

  function_name                     = aws_lambda_function.api.function_name
  provisioned_concurrent_executions = var.api_lambda_provisioned_concurrency
  qualifier                         = aws_lambda_alias.api_current.name
}

[sidekick-eimantas]

I ran into this and didn't want to rely on a fixed timeout, so instead I used a check-and-wait loop using the AWS CLI inside a null resource:

resource "null_resource" "wait_for_alias_not_have_weights" {
  triggers = {
    always_run = timestamp()
  }

  depends_on = [aws_lambda_alias.api_current]

  provisioner "local-exec" {
    # loop until the alias is not weighted
    command = <<EOF
    while : 
    do
      echo 'Checking for weights on API lambda alias...'
      weights=$(aws lambda get-alias --function-name ${aws_lambda_function.api.function_name} --name ${local.api_alias_name} --query 'RoutingConfig' || echo null)
      if [ "$weights" == "null" ]; then
        break
      fi
      sleep 5
    done
    EOF
  }
}

resource "aws_lambda_alias" "api_current" {
  name             = local.api_alias_name
  function_name    = aws_lambda_function.api.arn
  function_version = aws_lambda_function.api.version
}

resource "aws_lambda_provisioned_concurrency_config" "api" {
  depends_on = [null_resource.wait_for_alias_not_have_weights]

  function_name                     = aws_lambda_function.api.function_name
  provisioned_concurrent_executions = var.api_lambda_provisioned_concurrency
  qualifier                         = aws_lambda_alias.api_current.name
}

Worked beautifully, thank you

[jcourteauupgrade]

Perhaps there could be an optional flag on the aws_lambda_alias that causes the provider to wait until the alias's routing config converges to the provided one?

I'm running into this with a lambda which has its image (and thus version, and alias) updated by external deployment tooling, so I can't just add a time_sleep triggered by the alias version change (since it'd show up as drift whenever the alias is updated externally). I've created a terrible workaround but, like the various local-exec workarounds above, it's not really a good long-term option.

[spiderrockScrossan]

Experienced this issue today on aws provider v5.34.0. It seems to have fixed itself.. But I can't help but be concerned that it will come back.

[oscarrodar]

any update from on this?

[serozhenka]

For everyone curious, fixes in the following lines made @tgorton617 command work for me both in GitHub Actions and locally.

weights=$(aws lambda get-alias --function-name ${aws_lambda_function.api.function_name} --name ${local.api_alias_name} --query 'RoutingConfig' || echo "null")

if [ "$weights" = "null" ]; then

But still, I wish the proper solution had been available.