Lambda Deploy Function error: Resource is not in the state functionUpdated

[baumblatt]

Describe the bug

After the upgrade to version 1.12.0 (Latest) all the pipelines with Lambda functions deployment are ending with an error status even though the deployment was successful.

To reproduce

Just run pipeline with an task of type AWS Lambda Deploy Function in deployment mode codeonly.

Expected behavior

A successfully flag on finished of the task.

Error message

[error]Error: Error while updating function code: AccessDeniedException: Resource is not in the state functionUpdated

Your Environment

• On-prem or cloud based?: cloud based
• Azure DevOps version: N/A
• AWS Toolkit for Azure DevOps version: 1.12.0 (Latest)

Additional context

Everything was working fine prior the last update on Nov 17, 2021.

[ianByrneHaven]

Same issue here - didn't realise that it was still actually updating the function in AWS so that is good to know, cheers.

[weases]

Same here

[bryceitoc9]

Apologies for the delay, I'll take a look; this was added in the last release for the larger Lambda States rollout.

Just making sure, how long is this step taking to fail? Is it failing immediately or is it taking a few seconds after the code is deployed; also, is the function actually being updated in Lambda?

[ianByrneHaven]

It fails quite quickly - the entire task only takes 1s. And yes it is still updating the function code. Here are my full logs from a recent attempt:

Starting: LambdaDeployFunction
==============================================================================
Task         : AWS Lambda Deploy Function
Description  : General purpose deployment of AWS Lambda functions for all supported language runtimes.
Version      : 1.12.0
Author       : Amazon Web Services
Help         : Please refer to [AWS Lambda Developer Guide](https://docs.aws.amazon.com/lambda/latest/dg/) for more information on working with AWS Lambda.

More information on this task can be found in the [task reference](https://docs.aws.amazon.com/vsts/latest/userguide/lambda-deploy.html).

####Task Permissions
This task requires permissions to call the following AWS service APIs (depending on selected task options, not all APIs may be used):
* lambda:CreateFunction
* lambda:GetFunction
* lambda:UpdateFunctionCode
* lambda:UpdateFunctionConfiguration
==============================================================================
##[warning]This task uses Node 6 execution handler, which will be deprecated soon. If you are the developer of the task - please consider the migration guideline to Node 10 handler - https://aka.ms/migrateTaskNode10. If you are the user - feel free to reach out to the owners of this task to proceed on migration.
Configuring credentials for task
...configuring AWS credentials from service endpoint 'endpointId'
...endpoint defines standard access/secret key credentials
Configuring region for task
...configured to use region eu-west-1, defined in task.
Configuring credentials for task
...configuring AWS credentials from service endpoint 'endpointId'
...endpoint defines standard access/secret key credentials
Configuring region for task
...configured to use region eu-west-1, defined in task.
Updating code for function functionName
Waiting for function functionName to reach functionUpdated state...
##[error]Error: Error while updating function code: AccessDeniedException: Resource is not in the state functionUpdated
Finishing: LambdaDeployFunction

[ianByrneHaven]

And my pipelines yaml, if that helps:

- task: LambdaDeployFunction@1
  inputs:
    awsCredentials: ${{ Parameters.DevServiceConnection }}
    regionName: ${{ Parameters.DevAwsRegion }}
    deploymentMode: codeonly
    functionName: ${{ Parameters.DevFunctionName }}
    codeLocation: localfile
    localZipFile: "$(Pipeline.Workspace)/lambdaFunction/lambda_function.zip"

[bryceitoc9]

Thanks for that info @ianByrneHaven . If it's not too much to ask, what permissions does the role/user you're running as have? If that's too much to ask, do you have access to lambda.getFunction for all Lambdas?

[bryceitoc9]

Update: I have confirmed that this shows up if the service connection/role being used for the step doesn't have lambda.getFunction permissions. I'll see if I can roll out a new version soon with the following:

• If we can't match this specific AccessDeniedException's text, fail as usual (e.g. user doesn't have CreateFunction access)
• If we can match the AccessDeniedException's text, show a warning about the getFunction requirement and dumbly wait for 5 seconds
• If any subsequent calls fail since the function isn't ready, fail; the user should see the warning in the text later and adding getFunction permissions should resolve this going forward.

Everyone else here: try adding lambda.getFunction permissions to the service connection user/role and see if that fixes the error?

[weases]

I have double checked the permissions of the service connection and it has all the required permissions as stated here - including lambda:GetFunction.

[ianByrneHaven]

Yes I too have lambda:GetFunction on my service connection user, but scoped to the single function. Does it need to be scoped to all functions?

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Action": [
                "lambda:UpdateFunctionCode",
                "lambda:GetFunction"
            ],
            "Resource": "{functionArn}"
        }
    ]
}

[bryceitoc9]

Just making sure, you're getting the same AccessDeniedException: Resource is not in the state functionUpdated error? I'm trying to replicate by adding a permission at a time, and that seems like the crux (I have successfully gotten the wait to work with the exact ARN as well).

Additionally, are you using an Update Code Only deployment or are you updating the configuration as well?

[weases]

Just making sure, you're getting the same AccessDeniedException: Resource is not in the state functionUpdated error?

Yes

Additionally, are you using an Update Code Only deployment or are you updating the configuration as well?

I am using Update code only deployment

[bryceitoc9]

Ah, gotcha. Looks like you'll need `GetFunctionConfiguration" for update cases (I was only testing creating new functions from scratch). I'm planning on pushing out an update that can bypass that error message with a warning + a short timeout. I'll check in to get those docs updated as well.

Thanks for the info!

[weases]

Yep, I can confirm that with adding the GetFunctionConfiguration permission the error disappeared 🥳 Thx a lot @bryceitoc9

[ianByrneHaven]

Many thanks - I can also confirm that adding lambda:GetFunctionConfiguration to my IAM role has solved the issue for me.

[baumblatt]

Hello @bryceitoc9 ,

After added the permission lambda:GetFunctionConfiguration everything is working as espected.

Should we close this?

Best regards, Bernardo.

[bryceitoc9]

Thanks all for confirming! I'll close it after I merge https://github.com/aws/aws-toolkit-azure-devops/pull/448 .