Amplify CICD - Security token expired

francoisharmse777 commented 1 year ago

Before opening, please confirm:

[X] I have checked to see if my question is addressed in the FAQ.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
[X] I have removed any sensitive information from my code snippets and submission.

App Id

dpdexyuc6dp7b

AWS Region

eu-west-1

Amplify Hosting feature

Backend builds, Build settings, Service role

Describe the bug

An Amplify build is triggered through the Hosting Environment integration with Gitlab, and committing to the relevant branch which is connected to this amplify app environment, Usually, this works perfectly - and upon a commit, a build is triggered.

Recently the builds fail because a security token expires.

Amplify 10.4.0 Security Token Expired after 1 our

The Amplify Build timeout is set to 180mins. Yet the security token expires, and eventually, the Amplify CICD build fails.

This is when Amplify builds are triggered through the git auto build through hosting environments integration.

I already tried to increase the Amplify service role session time to 4hours. Yet, the security Token expires after 1 hour.

Logs Build started 2022-11-08T09:32:11.428Z [INFO]: Git SSH Key acquired Error (timeout occurred) 2022-11-08T10:32:33.533Z [INFO]: 🛑 The security token included in the request is expired Learn more at: https://docs.amplify.aws/cli/project/troubleshooting/ build finished (with failure) 2022-11-08T11:31:36.472Z [WARNING]: ✖ Sending zip 2022-11-08T11:31:37.277Z [INFO]: 🛑 The security token included in the request is expired Learn more at: https://docs.amplify.aws/cli/project/troubleshooting/ 2022-11-08T11:31:37.277Z [WARNING]: - Creating Zip 2022-11-08T11:31:37.281Z [INFO]: 🛑 The security token included in the request is expired Learn more at: https://docs.amplify.aws/cli/project/troubleshooting/ 2022-11-08T11:31:37.281Z [WARNING]: - Creating Zip 2022-11-08T11:31:37.285Z [INFO]: 🛑 The security token included in the request is expired Learn more at: https://docs.amplify.aws/cli/project/troubleshooting/ 2022-11-08T11:31:37.286Z [WARNING]: - Creating Zip 2022-11-08T11:31:37.289Z [INFO]: 🛑 The security token included in the request is expired Learn more at: https://docs.amplify.aws/cli/project/troubleshooting/ 2022-11-08T11:31:37.289Z [WARNING]: - Creating Zip 2022-11-08T11:31:37.342Z [INFO]: ✅ Report saved: /tmp/clientname/report-1667907097281.zip 2022-11-08T11:31:37.342Z [WARNING]: - Sending zip 2022-11-08T11:31:37.348Z [INFO]: ✅ Report saved: /tmp/clientname/report-1667907097284.zip 2022-11-08T11:31:37.348Z [WARNING]: - Sending zip 2022-11-08T11:31:37.353Z [INFO]: ✅ Report saved: /tmp/clientname/report-1667907097293.zip 2022-11-08T11:31:37.353Z [WARNING]: - Sending zip 2022-11-08T11:31:37.356Z [INFO]: ✅ Report saved: /tmp/clientname/report-1667907097289.zip 2022-11-08T11:31:37.356Z [WARNING]: - Sending zip 2022-11-08T11:31:37.509Z [INFO]:

Expected behavior

Amplify build should run through all the relevant stages such as Backend (Prebuild, build) and Frontend (prebuild and build) phases. The build complete successfully, building the backend environment (running all the relevant cloudformation stacks) and also deploying the frontend react application.

Reproduction steps

rerun amplify build or commit a new change to the git branch connected to the amplify environment

Build Settings

version: 1
backend:
  phases:
    preBuild:
      commands:                
        - yum -y install jq wget        
        - AWS_ACCOUNT=`aws sts get-caller-identity | jq -r '.Account'`        
        - sed "s/SOME_AWS_ACCOUNT/${AWS_ACCOUNT}/g; s/ENV/${USER_BRANCH}/g; s/AMPLIFY_APP_ID/${AWS_APP_ID}/g" _CI/scripts/congito-config.ts > amplify/backend/auth/tagetclient*/congito-config.ts       
    build:
      commands:        
        - echo --------- SLACK NOTIFICATION ----------      
        - . _CI/scripts/slack-start.sh          
        - amplifyPush
frontend:
  phases:
    preBuild:
      commands:
        - yum -y install jq wget
        - echo ======================
        - echo AWS_APP_ID = $AWS_APP_ID
        - echo AWS_JOB_ID = $AWS_JOB_ID
        - echo ======================
        - npm ci
        - RC=$?
        - if [ $RC -eq 0 ] 
        - then
        - echo "npm ci Succeeded"
        - else 
        - . _CI/scripts/slack-fe-failed.sh
        - fi
        - amplify status | egrep "GraphQL endpoint"
        - amplify status | egrep "GraphQL API KEY" | awk '{print $4}'
        - GRAPHQL_ENDPOINT=`amplify status | egrep "GraphQL endpoint" | awk '{print $3}'`        
        - GRAPHQL_API_KEY=`amplify status | egrep "GraphQL API KEY" | awk '{print $4}'`
        - S3_DOCU_BUCKET=`aws ssm get-parameter --name "s3-bucket-name" | jq -r '.Parameter.Value'`
        - RC=$?
        - if [ $RC -eq 0 ] 
        - then
        - echo "Retrieved GraphQL Endpoint from amplify"
        - else 
        - . _CI/scripts/slack-fe-failed.sh
        - fi        
        - COGNITO_USERPOOL_ID=`aws cognito-idp list-user-pools --max-results 20 | jq -r '.UserPools[] | select(.Name | contains("clientname")).Id'`
        - COGNITO_IDENTITY_ID=`aws cognito-identity list-identity-pools --max-results 20 | jq -r '.IdentityPools[] | select(.IdentityPoolName | contains("clientname")).IdentityPoolId'`
        - COGNITO_USERPOOL_WEBCLIENTID=`aws cognito-idp list-user-pool-clients --user-pool-id  $COGNITO_USERPOOL_ID | jq -r '.UserPoolClients[] | select(.ClientName | contains("Web")).ClientId'`        
        - echo "------------------------"
        - cat .env.$USER_BRANCH
        - echo "------------------------" 
    build:
      commands:
        - npm run build:$USER_BRANCH
        - echo --------- SLACK NOTIFICATION ----------
        - . _CI/scripts/slack.sh
    postBuild:
      commands:
        - DOMAIN=`aws amplify list-apps | jq -r '.apps[0].defaultDomain'`
        - aws configure set cli_follow_urlparam false
        - aws ssm put-parameter --name "parmname1" --value "https://${USER_BRANCH}.${DOMAIN}" --type String --overwrite
        - COGNITO_USERPOOL_ID=`aws cognito-idp list-user-pools --max-results 20 | jq -r '.UserPools[] | select(.Name | contains("someclientname")).Id'`
        - COGNITO_IDENTITY_ID=`aws cognito-identity list-identity-pools --max-results 20 | jq -r '.IdentityPools[] | select(.IdentityPoolName | contains("someclientname")).IdentityPoolId'`
        - COGNITO_USERPOOL_WEBCLIENTID=`aws cognito-idp list-user-pool-clients --user-pool-id  $COGNITO_USERPOOL_ID | jq -r '.UserPoolClients[] | select(.ClientName | contains("Web")).ClientId'`
        - aws ssm put-parameter --name "cognito-userpool-id" --value "${COGNITO_USERPOOL_ID}" --type String --overwrite
        - DOCUMENTS_BUCKET=`aws s3 ls | grep someclientname_bucket | awk '{print $3}'`
        - aws ssm put-parameter --name "documents-bucket" --value "${DOCUMENTS_BUCKET}" --type String --overwrite
  artifacts:
    baseDirectory: /build
    files:
      - '**/*'
  cache:
    paths: []

Log output

``` # Put your logs below this line ```

Additional information

Amplify includes the following services / components Category │ Resource name │ Operation │ Provider plugin │ │ Function │ someclientnamePostConfirmation │ Update │ awscloudformation │ │ Function │ AdminQueries6c95c236 │ Update │ awscloudformation │ │ Function │someclientnameCustomMessage │ Update │ awscloudformation │ │ Auth │ someclientname66c6708a │ Update │ awscloudformation │ │ Auth │ userPoolGroups │ Update │ awscloudformation │ │ Api │ someclientname │ Update │ awscloudformation │ │ Api │ AdminQueries │ Update │ awscloudformation │ │ Storage │ someclientnamedocumentsbucket │ Update │ awscloudformation │ │ Geo │ placeindex2bd039b4 │ Update │ awscloudformation │ │ Geo │ placeindex0517f685 │ Update │ awscloudformation │ │ Geo │ map94417b45 │ Update │ awscloudformation

Also using an override on the AUTH service - to customise Cognito email functionality and messages.

ghost commented 1 year ago

Hi @francoisharmse777 👋🏽 we apologize for this inconvenience. Currently, the security token has a hard limit expiration of 60 minutes. We understand this is a blocker for customers who have backend deployments that require an extended build time and we are tracking this behavior internally for prioritization.

ghost commented 1 year ago

Closing as a duplicate of https://github.com/aws-amplify/amplify-hosting/issues/2751

github-actions[bot] commented 1 year ago

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.

github-actions[bot] commented 1 year ago

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

aws-amplify / amplify-hosting