jenkinsci / pipeline-aws-plugin

Jenkins Pipeline Step Plugin for AWS
https://plugins.jenkins.io/pipeline-aws/
Apache License 2.0
431 stars 202 forks source link

invokeLambda method causes Lambda to be executed four times #208

Open JPGrieb opened 4 years ago

JPGrieb commented 4 years ago

Description

I’m using the "invokeLambda" as part of my Jenkins pipeline to kick off a Lambda function that is taking approximately 50 to 70 seconds to complete.

The lambda function completes successfully each time.

The Lambda function is consistently getting executed four times instead of just once even though I am calling the “invokeLambda” method only once.

Each execution has a different execution id so it does not appear to be AWS retrying the function.

Steps to Reproduce

withAWS(region: "eu-west-1", roleAccount: '906291453955', role: 'a205470-JenkinsCrossAccount') {

     invokeLambda(functionName: 'a205470-wcol-update-monitors',
    payload: [ "environment" : "ppe02", "bucket" : "preprod-ppe02", "method" : "Create" ]))

}

Expected behavior: Lamba would be invoked once

Actual behavior: Lambda was invoked 4 times

Environment

Jenkins-Version: Jenkins ver. 2.190.2

Java-Version: ????

Plugin-Version: 1.36 Master/Slave Setup: No

rafaeldalsenter commented 3 years ago

I have a difficulty similar to the @JPGrieb report

When invoking a Lambda function using the invokeLambda method, it's not waiting for the execution to return. In my case I'm doing this:

Jenkinspipeline:

stage('AWS Lambda'){
    steps {
        invokeLambda([awsRegion: 'us-east-1',
            functionName: 'function-lambda', 
            payload: "{ \"Suspend\": \"true\"}] }", 
            synchronous: true, 
            useInstanceCredentials: true])
    }
}

Lambda function code:

exports.handler = async (event, context) => {
    return new Promise(
        function(resolve, reject) {
            execMethod({ ... }, function(err, data) {
                if(err){
                    reject(Error(err));
                }else{
                    resolve("Done!");
                }
            })
        }
    )
}

Jenkins version: 1.24.3

Plugin version: 1.43

aditya-kapre-shrewsbury commented 3 years ago

@rafaeldalsenter Were you able to address the issue and wait for execution of lambda function to complete ? If yes, how did you achieve this ?

rafaeldalsenter commented 3 years ago

Hi @adityakapre,

To solve this, I installed AWS CLI and used sh to invoke the lambda function:

aws lambda invoke --function-name function-lambda --payload \"{ \"Suspend\": \"true\"}] }\" --invocation-type RequestResponse response.json
tony-bondarenko commented 3 years ago

The reason why you see this behavior is cli\sdk retries: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-retries.html The issue can be reproduced in AWS CLI as well. If you have a long-running lambda (that runs more than 60 seconds), after 60 seconds first call will be treated as time-outed and will be repeated AWS_MAX_ATTEMPTS -1 times. To fix it for CLI, you need to run invoke command as follows: AWS_MAX_ATTEMPTS=1 aws lambda invoke --cli-read-timeout 300 ... This call will invoke lambda exactly one time and will error with a timeout after 5 minutes. Same configuration for retries can be done for Java SDK: clientConfiguration.setMaxErrorRetry(0) Unfortunately, I don't have time to raise a PR, but hope this comment helps.

MedMokhtarAmmar commented 1 year ago

is this problem solved yet or i need to use a sh script to invoke lambda ?