Get latest version number of data source aws_lambda_function

ghost commented 5 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

To associate a lambda function to CloudFront distribution as a Lambda@Edge (using a cache behiavor's lambda_function_association block), it is required to provide a numbered version. The qualifier $LATEST will (unfortunately) not work.

At the moment, that data source allow to pass a qualifier as an argument, but its qualified_arn attribute will reference that exact same qualifier, not necessarily a version number, unless the attribute itself is a version number, but that requires that version number to be known.

From what I understand, the AWS SDK does not provide a built-in function to get the last published version directly, but as a workaround, it can be retrieved using the pagination API, such as shown in the resource aws_lambda_function: https://github.com/terraform-providers/terraform-provider-aws/blob/v2.27.0/aws/resource_aws_lambda_function.go#L583

New or Affected Resource(s)

data-source aws_lambda_function

Proposal: add new attributes to the data source aws_lambda_function to access the latest numbered version (and/or even better: the fully qualified latest numbered version?)

latest_version_number (the actual version number only)
latest_version_qualified_arn (the fully qualified latest numbered version of the function)

(better names can probably be found)

Potential Terraform Configuration

This is an example how what would be possible with such a feature:

data "aws_lambda_function" "example_lambda" {
  function_name = "example"
  qualifier     = "$LATEST"
}

resource "aws_cloudfront_distribution" "example_distribution" {
  # ...

  default_cache_behavior {
    # ...

    lambda_function_association {
      event_type   = "viewer-request"
      # The following ARN MUST BE a numbered version
      # Option 1: format the ARN manually
      lambda_arn   = format("%s:%s", data.aws_lambda_function.example_lambda.arn, data.aws_lambda_function.example_lambda.latest_version_number)
      # Option 2: use a fully qualified ARN if provided
      lambda_arn   = data.aws_lambda_function.example_lambda.latest_version_qualified_arn
    }
  }
}

References

https://github.com/terraform-providers/terraform-provider-aws/issues/8782 (I initially posted a comment on that issue)
https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-requirements-limits.html

If the change can be resumed to something as simple as copy-pasting the behavior of the resource to the data source, I would be more than happy to create a PR for it myself.

I would like to get some feedback on the proposal before starting, though.

And since I have never written a single line of Go code in my life before, I would need some guidance regarding contribution and especially testing, if that is possible?

ghost commented 4 years ago

(Hello, I am deleting this account in favour of @flo-sch, will keep watching it from there.)

@ddriddle I saw that your PR has been merged, is that a workaround that can already be used?

ddriddle commented 4 years ago

@flo-sch Yes, setting an alias called latest in Terraform code works fine as a workaround for CloudFront.

brucedvgw commented 4 years ago

Has there been any progress on the data resource for this? I'm keen not to hardcode the version number if I can help it.

nkoterba commented 4 years ago

I attempted the suggest workarounds in https://github.com/terraform-providers/terraform-provider-aws/issues/8782 but none of them seem to work since I can't control the lambda and do not publish it.

Has anybody found a workaround if you're not publishing the lambda using data either on aws_lambda_alias or aws_lambda_function?

Hasgaroth commented 4 years ago

I could really do with a fix for this issue too, although as I do publish the Lambda functions via another Terraform run, I am able to extract the latest version number that was deployed by Terraform using the following code (edited to remove some specifics).

data "terraform_remote_state" "lambda" {
  backend = "s3"

  config = {
    encrypt  = true
    bucket   = "terraform-state"
    key      = "${local.vpc_tags["Account"]}/compute/lambda/global.tfstate"
    region   = "eu-west-2"
    role_arn = "arn:aws:iam::123456789012:role/terraform"
  }
}

data "aws_lambda_function" "cloudfront" {
  provider      = aws.cf_cert_region      # us-east-1
  function_name = "function_name"
  qualifier     = data.terraform_remote_state.lambda.outputs.lambda_details["function_name"].version
}

I hope this helps someone until this fix gets landed.

pzanuto commented 4 years ago

I could really do with a fix for this issue too, although as I do publish the Lambda functions via another Terraform run, I am able to extract the latest version number that was deployed by Terraform using the following code (edited to remove some specifics).
data "terraform_remote_state" "lambda" {
  backend = "s3"

  config = {
    encrypt  = true
    bucket   = "terraform-state"
    key      = "${local.vpc_tags["Account"]}/compute/lambda/global.tfstate"
    region   = "eu-west-2"
    role_arn = "arn:aws:iam::123456789012:role/terraform"
  }
}

data "aws_lambda_function" "cloudfront" {
  provider      = aws.cf_cert_region      # us-east-1
  function_name = "function_name"
  qualifier     = data.terraform_remote_state.lambda.outputs.lambda_details["function_name"].version
}
I hope this helps someone until this fix gets landed.

Thank you! It works fine for me... I just had to include an output after the lambda resource is created

BertrandMarechal commented 3 years ago

I arrived here by looking for how to set up this with a resource, not data. Here is what I found if someone else follows this path too

resource "aws_lambda_function" "example_lambda" {
  # ...
  publish          = true // This is an important part, it publishes the lambda and creates a version
}

resource "aws_cloudfront_distribution" "example_distribution" {
  # ...

  default_cache_behavior {
    # ...

    lambda_function_association {
      event_type   = "viewer-request"
      # The following ARN MUST BE a numbered version
      lambda_arn   = "${aws_lambda_function.example_lambda.arn}:${aws_lambda_function.example_lambda.version}"
    }
  }
}

That works like a charm.

Woitekku commented 3 years ago

My workaround:

One terraform:

resource "aws_lambda_alias" "basic-auth" {
    provider = aws.us-east-1
    name = "latest"
    function_name = aws_lambda_function.basic-auth.arn
    function_version = aws_lambda_function.basic-auth.version
}

Completely different terraform/different state in the same infra:

data "aws_lambda_alias" "basic-auth" {
  provider = aws.us-east-1
  function_name = "basic-auth"
  name          = "latest"
}

data "aws_lambda_function" "basic-auth" {
  provider = aws.us-east-1
  function_name = "basic-auth"
  qualifier = data.aws_lambda_alias.basic-auth.function_version
}

Works like a charm...

flo-sch commented 3 years ago

I guess those workaround only work when publishing the lambda with Terraform though?

I might have missed to mention that, but in my initial use-case, Lambda functions were published by something else (serverless framework), hence the proposal for a new data that could look up the latest version, no matter what was used to deploy it

Woitekku commented 3 years ago

The deployment method has nothing to do with it. You can even clickops it, then use terraform data to read what has been there - you have to read alias and the function, all together, combined. Just make sure you do an alias, in my case I used terraform resource to do it, but afaik SLS/SAM has the same functionality.

flo-sch commented 3 years ago

Oh nice, I missed that, then that sounds like a decent workaround indeed!

tymik commented 3 years ago

My workaround:

One terraform:

resource "aws_lambda_alias" "basic-auth" {
    provider = aws.us-east-1
    name = "latest"
    function_name = aws_lambda_function.basic-auth.arn
    function_version = aws_lambda_function.basic-auth.version
}

Completely different terraform/different state in the same infra:

data "aws_lambda_alias" "basic-auth" {
  provider = aws.us-east-1
  function_name = "basic-auth"
  name          = "latest"
}

data "aws_lambda_function" "basic-auth" {
  provider = aws.us-east-1
  function_name = "basic-auth"
  qualifier = data.aws_lambda_alias.basic-auth.function_version
}

Works like a charm...

This is perfect! Easiest solution to implement for this case!

sriniavgs commented 2 years ago

@tymik @Woitekku I tried the same steps, but the qualifier still points to one version older

data "aws_lambda_alias" "current" { function_name = module.lambda_function.function_name name = "current" }

resource "aws_lambda_provisioned_concurrency_config" "lambda_warming" { count = var.provision_concurrency_flag ? 1 : 0 function_name = module.lambda_function.function_name provisioned_concurrent_executions = var.provision_executions qualifier = data.aws_lambda_alias.current.function_version }

--> the above code sets the provisioned concurrency on the previous version. (example: "alias: current" --> v5, concurrency set on v4)

I also tried this and its same behavior, concurrency set on previous version. qualifier = aws_lambda_function.main.version (using tf resource object, instead of tf module)

Any suggestions?

tymik commented 2 years ago

@sriniavgs it seems that you are missing the data "aws_lambda_function configuration block, can't test now but I'd say it is required for whole workaround to work. The another thing is - do you have the current alias for your lambda and is it updated? In the example I am using alias latest which is there by default, provided by AWS - if you use custom alias, you need to take care of it on your own - custom alias doesn't necessarily have to point to the latest version of your lambda function.

You might also want to check AWS docs regarding Lambda versions and aliases

jamiejackson commented 2 years ago

Update: In my particular case, the versioned ARN is available as an output from the CloudFormation, so I'm set.

I'm trying @Woitekku 's workaround. The function was generated by a CloudFront template (via Terraform), so I don't think I have any more direct access to it as a resource.

data "aws_lambda_alias" "cognito_auth__check_auth_handler__localdev" {
  provider = aws.us-east-1
  function_name = "serverlessrepo-localdevCloudfront-CheckAuthHandler-yCGyEJodo4jh"
  name          = "latest"
}

data "aws_lambda_function" "cognito_auth__check_auth_handler__localdev" {
  provider = aws.us-east-1
  function_name = "serverlessrepo-localdevCloudfront-CheckAuthHandler-yCGyEJodo4jh"
  qualifier = data.aws_lambda_alias.cognito_auth__check_auth_handler__localdev.function_version
}

However, Terraform throws this, which I don't know how to interpret:

│ Error: Provider configuration not present
│ 
│ To work with data.aws_lambda_alias.cognito_auth__check_auth_handler__localdev its original provider configuration at provider["registry.terraform.io/hashicorp/aws"].us-east-1 is required, but it has been removed. This occurs when
│ a provider configuration is removed while objects created by that provider still exist in the state. Re-add the provider configuration to destroy data.aws_lambda_alias.cognito_auth__check_auth_handler__localdev, after which you
│ can remove the provider configuration again.

Do you know what this means and how to solve it?

tymik commented 2 years ago

@jamiejackson you need to define an additional provider for provider = aws.us-east-1 in your resource to work. a good practice is to have a separate one for this particular region as it is the region of the CloudFront and you need to set it up in that region, which may not necessarily be valid for other resources you spin up.

here's a snippet:

provider "aws" {
  alias  = "us-east-1"
  region = "us-east-1"
}

github-actions[bot] commented 2 years ago

This functionality has been released in v4.29.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

github-actions[bot] commented 2 years ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

hashicorp / terraform-provider-aws