Failed headless amplify pull does not return error code

valentinrelief commented 2 years ago

Before opening, please confirm:

[X] I have installed the latest version of the Amplify CLI (see above), and confirmed that the issue still persists.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
[X] I have removed any sensitive information from my code snippets and submission.

How did you install the Amplify CLI?

npm install -g @aws-amplify/cli

If applicable, what version of Node.js are you using?

No response

Amplify CLI Version

10.0.0

What operating system are you using?

Ubuntu (GitHub Actions)

Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.

n/a

Amplify Categories

Not applicable

Amplify Commands

pull

Describe the bug

I'm trying to setup a GitHub Action workflow to automatically do an amplify pull and push on each commit, so that devs do not have to remember to do this on each commit.

When running amplify pull ... I'm currently getting the error:

Invalid configuration settings!
? Do you want to retry configuration? (y/N)

But the script is not exiting because amplify pull is not returning an error status code, even though it errored.

This is causing the script to move onto the next step, which is unexpected.

Here's my workflow:

# This is a basic workflow to help you get started with Actions

name: Run Integration Tests

# Controls when the workflow will run
on:
  # Triggers the workflow on push request events for the "preprod" branch
  push:
    branches: [ "preprod" ]
#  pull_request:
#    branches: [ "main" ]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # This workflow contains a single job called "build"
  build:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest
    # These permissions are needed to interact with GitHub's OIDC Token endpoint.
    permissions:
      id-token: write
      contents: read
    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v4
        with:
          python-version: '3.10'
          cache: 'pip' # caching pip dependencies
      # Integration test expects ${AMPLIFY_STAGE} to be set, so we get that from the branch name
      # https://stackoverflow.com/a/58035262
      - name: Extract branch name
        shell: bash
        run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF#refs/heads/})"
        id: extract_branch
      # Set the env variable https://tinyurl.com/2p8tjpeu
      - name: Set AMPLIFY_STAGE env value
        run: echo "AMPLIFY_STAGE=${{ steps.extract_branch.outputs.branch }}" >> $GITHUB_ENV
      # Setup with https://www.automat-it.com/post/using-github-actions-with-aws-iam-roles
      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          role-to-assume: arn:aws:iam::[redacted]:role/github-actions-role
          aws-region: us-east-1
      - name: Amplify deploy
        run: |
          npm install -g @aws-amplify/cli
          ./scripts/headless_amplify_pull_push.sh
        env:
          AMPLIFY_APP_ID: [redacted]
      - name: Install dependencies (from Pipfile using pipenv) and run integration tests
        run: |
          pip install pipenv
          cd src/integration_tests
          pipenv install
          pipenv run python -m pytest .

And here is my script:

#!/bin/bash
set -eu # set e quits on first error, u errors if any variable is unset
IFS='|'
# This script is the headless equivalent of:
#   amplify pull --appId $AMPLIFY_APP_ID --envName $AMPLIFY_STAGE
# Meant to be used within GitHub Actions

# The following env vars are expected to be set:
# AMPLIFY_APP_ID: [redacted]
# AMPLIFY_STAGE: preprod
# AWS_REGION: us-east-1
# AWS_ACCESS_KEY_ID: ***
# AWS_SECRET_ACCESS_KEY: ***

# The last three AWS_ variables are set by aws-actions/configure-aws-credentials

# Verify all vars are set. set -u ensures these error and stop the script
: "${AMPLIFY_STAGE?Need to set AMPLIFY_STAGE}"
: "${AWS_REGION?Need to set AWS_REGION}"
: "${AWS_ACCESS_KEY_ID?Need to set AWS_ACCESS_KEY_ID}"
: "${AWS_SECRET_ACCESS_KEY?Need to set AWS_SECRET_ACCESS_KEY}"

echo "Working with branch/env ${AMPLIFY_STAGE}"

REACTCONFIG="{\
\"SourceDir\":\"src\",\
\"DistributionDir\":\"dist\",\
\"BuildCommand\":\"npm run-script build\",\
\"StartCommand\":\"npm run-script start\"\
}"
FRONTEND="{\
\"frontend\":\"javascript\",\
\"framework\":\"none\",\
\"config\":$REACTCONFIG\
}"

AWSCLOUDFORMATIONCONFIG="{\
\"configLevel\":\"general\",\
\"useProfile\":false,\
\"accessKeyId\":\"$AWS_ACCESS_KEY_ID\",\
\"secretAccessKey\":\"$AWS_SECRET_ACCESS_KEY\",\
\"region\":\"$AWS_REGION\"\
}"

AMPLIFY="{\
\"projectName\":\"ReliefBackend\",\
\"appId\":\"$AMPLIFY_APP_ID\",\
\"envName\":\"$AMPLIFY_STAGE\",\
\"defaultEditor\":\"none\"\
}"

PROVIDERS="{\
\"awscloudformation\":$AWSCLOUDFORMATIONCONFIG\
}"

echo "running: amplify pull..."

amplify pull \
--amplify $AMPLIFY \
--frontend $FRONTEND \
--providers $PROVIDERS \
--yes

### THE PREVIOUS STEP FAILS but it keeps going ###

CODEGEN="{\
\"generateCode\":true,\
\"codeLanguage\":\"javascript\",\
\"fileNamePattern\":\"src/graphql/**/*.js\",\
\"generatedFileName\":\"API\",\
\"generateDocs\":true\
}"

echo "running: amplify push..."
amplify push \
--codegen $CODEGEN \
--yes

Expected behavior

When commands have an error, they should return an error code.

It would also be great if the error gave more information than "Invalid configuration settings!" (I've been trying to debug this for days now. It runs fine locally)

Reproduction steps

run a failing amplify pull and observe the status code

GraphQL schema(s)

```graphql # Put schemas below this line ```

Project Identifier

No response

Log output

``` Run npm install -g @aws-amplify/cli added 26 packages, and audited 27 packages in 9s 7 packages are looking for funding run `npm fund` for details found 0 vulnerabilities Working with branch/env preprod running: amplify pull... Invalid configuration settings! ? Do you want to retry configuration? (y/N) running: amplify push... 🛑 An error occurred during the push operation: / File at path: '/home/runner/work/backend/backend/amplify/.config/project-config.json' does not exist ⚠️ Review the Amplify CLI troubleshooting guide for potential next steps: https://docs.amplify.aws/cli/project/troubleshooting/ ```

Additional information

No response

josefaidt commented 2 years ago

Hey @valentinrelief :wave: thanks for raising this! I have a few notes:

When working with initializing the project, amplify init may be a suitable replacement for amplify pull: https://github.com/josefaidt/amplify-with-github-actions/blob/main/.github/workflows/publish-amplify-app.yml

But the script is not exiting because amplify pull is not returning an error status code, even though it errored.

From the codebase I see where if there is not a confirmation on the retry prompt the CLI would correctly error out with a status code of 1, however it appears given the CI setting and the use of --yes it is instead auto-confirming the retry prompt and returning an empty initialization https://github.com/aws-amplify/amplify-cli/blob/dev/packages/amplify-provider-awscloudformation/src/configuration-manager.ts#L242 -- this is a miss in the behavior when considering a CI setting

From your sample snippets, is AMPLIFY_APP_ID being set properly?

valentinrelief commented 2 years ago

Thanks for the reply. Some of the snippets got a bit jumbled as I was trying to redact some private info, so here's what we're actually using.

BTW, I was able to trace the issue to the fact that amplify pull doesn't use the AWS_ env variables and instead only seems to read from the ~/.aws/ files for info.

integ_tests.yml

# This is a basic workflow to help you get started with Actions

name: Run Integration Tests

# Controls when the workflow will run
on:
  # Triggers the workflow on push request events for the "preprod" branch
  push:
    branches: [ "preprod" ]
#  pull_request:
#    branches: [ "main" ]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # This workflow contains a single job called "build"
  build:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest
    # These permissions are needed to interact with GitHub's OIDC Token endpoint.
    permissions:
      id-token: write
      contents: read
    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
      - uses: actions/checkout@v3
      # Integration test expects ${AMPLIFY_STAGE} to be set, so we get that from the branch name
      # https://stackoverflow.com/a/58035262
      - name: Extract branch name
        shell: bash
        run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF#refs/heads/})"
        id: extract_branch
      # Set the env variable https://tinyurl.com/2p8tjpeu
      - name: Set AMPLIFY_STAGE env value
        run: echo "AMPLIFY_STAGE=${{ steps.extract_branch.outputs.branch }}" >> $GITHUB_ENV
      # Setup with https://www.automat-it.com/post/using-github-actions-with-aws-iam-roles
      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          role-to-assume: ${{ secrets.AWS_GITHUB_ACTIONS_ROLE }}
          aws-region: us-east-1
      - name: Amplify deploy
        run: |
          npm install -g @aws-amplify/cli
          ./scripts/headless_amplify_pull_push.sh
        env:
          AMPLIFY_APP_ID: ${{ secrets.AMPLIFY_APP_ID }}
      - uses: actions/setup-python@v4
        with:
          python-version: '3.9'
          cache: 'pip' # caching pip dependencies
      - name: Install dependencies (from Pipfile using pipenv) and run integration tests
        run: |
          pip install pipenv
          cd src/integration_tests
          pipenv install
          pipenv run python -m pytest .

headless_amplify_pull_push.sh

#!/bin/bash
set -eu # set e quits on first error, u errors if any variable is unset
IFS='|'
# This script is the headless equivalent of:
#   amplify pull --appId $AMPLIFY_APP_ID --envName $AMPLIFY_STAGE
# Meant to be used within GitHub Actions

# The following env vars are expected to be set:
# AMPLIFY_APP_ID: d39y0000000000
# AMPLIFY_STAGE: preprod
# AWS_REGION: us-east-1
# AWS_ACCESS_KEY_ID: ***
# AWS_SECRET_ACCESS_KEY: ***
# The last three AWS_ variables are set by aws-actions/configure-aws-credentials

# Verify all vars are set. set -u ensures these error and stop the script
: "${AMPLIFY_APP_ID?Need to set AMPLIFY_APP_ID}"
: "${AMPLIFY_STAGE?Need to set AMPLIFY_STAGE}"
: "${AWS_REGION?Need to set AWS_REGION}"
: "${AWS_ACCESS_KEY_ID?Need to set AWS_ACCESS_KEY_ID}"
: "${AWS_SECRET_ACCESS_KEY?Need to set AWS_SECRET_ACCESS_KEY}"

echo "Working with branch/env ${AMPLIFY_STAGE}"

# amplify doesn't support session tokens like the rest of AWS, so we have manually
# construct ~/.aws files:
# https://github.com/aws-amplify/amplify-cli/issues/7642#issuecomment-875881203

if [[ -n $CI && -n $AWS_ACCESS_KEY_ID && -n $AWS_SECRET_ACCESS_KEY ]]; then
    mkdir -p ~/.aws

    AWS_PROFILE=cicd
    CONFIG_PATH=~/.aws/config
    CREDENTIALS_PATH=~/.aws/credentials

cat <<-EOF >> $CONFIG_PATH
    [profile $AWS_PROFILE]
    region=$AWS_REGION
    output=json
EOF

cat <<-EOF >> $CREDENTIALS_PATH
    [$AWS_PROFILE]
    aws_access_key_id=$AWS_ACCESS_KEY_ID
    aws_secret_access_key=$AWS_SECRET_ACCESS_KEY
EOF
    if [[ -n $AWS_SESSION_TOKEN ]]; then
        echo "aws_session_token=$AWS_SESSION_TOKEN" >> $CREDENTIALS_PATH
    fi
fi

# https://docs.amplify.aws/cli/usage/headless/#sample-script-3

REACTCONFIG="{\
\"SourceDir\":\"src\",\
\"DistributionDir\":\"dist\",\
\"BuildCommand\":\"npm run-script build\",\
\"StartCommand\":\"npm run-script start\"\
}"
FRONTEND="{\
\"frontend\":\"javascript\",\
\"framework\":\"none\",\
\"config\":$REACTCONFIG\
}"

AWSCLOUDFORMATIONCONFIG="{\
\"configLevel\":\"general\",\
\"useProfile\":true,\
\"profileName\":\"${AWS_PROFILE:-default}\",\
\"region\":\"$AWS_REGION\"\
}"

AMPLIFY="{\
\"projectName\":\"ReliefBackend\",\
\"appId\":\"$AMPLIFY_APP_ID\",\
\"envName\":\"$AMPLIFY_STAGE\",\
\"defaultEditor\":\"none\"\
}"

PROVIDERS="{\
\"awscloudformation\":$AWSCLOUDFORMATIONCONFIG\
}"

echo "running: amplify pull..."

amplify pull \
--amplify $AMPLIFY \
--frontend $FRONTEND \
--providers $PROVIDERS \
--yes

CODEGEN="{\
\"generateCode\":true,\
\"codeLanguage\":\"javascript\",\
\"fileNamePattern\":\"src/graphql/**/*.js\",\
\"generatedFileName\":\"API\",\
\"generateDocs\":true\
}"

echo "running: amplify push..."
amplify push \
--codegen $CODEGEN \
--yes

josefaidt commented 2 years ago

BTW, I was able to trace the issue to the fact that amplify pull doesn't use the AWS_ env variables and instead only seems to read from the ~/.aws/ files for info.

Great callout! It appears there are two bugs here:

CLI should exit when configuration is invalid in a CI setting (where we're using --yes or when CI=true) instead of prompting
CLI should also use AWS_ environment variables when pulling

As a workaround, can you try changing pull to init? We likely do not need the additional information such as the project config for init

valentinrelief commented 2 years ago

Sorry if I wasn't clear: the amplify pull command runs successfully once I got the aws credentials stored in the ~/.aws file. As a workaround, I'm creating the ~/.aws/config and credentials file

So currently, the only errors are what you listed above: 1 and 2

valentinrelief commented 2 years ago

I suppose a tertiary issue was that the error message Invalid configuration settings! wasn't very helpful. It took a while to narrow it down to the credentials.

wlee221 commented 1 year ago

Amplify UI team also found this issue while setting up headless pull with GitHub Actions.

I did some root causing, and found that AWS_SESSION_TOKEN specifically were ignored in headless pull flow. I could confirm that awsConfigInfo.config.sessionToken was undefined in

https://github.com/aws-amplify/amplify-cli/blob/9f577b9705e56e0f5e0dd0d043805f4037295801/packages/amplify-provider-awscloudformation/src/configuration-manager.ts#L467

Other AWS env variables were set correctly. I could also confirm that setting sessionToken to AWS_SESSION_TOKEN in that file fixes the error.

We are currently using the workaround @valentinrelief uses, but it'd be great if this can be solved from CLI end so that we can just use setup-aws-credentials without additional setups.

aws-amplify / amplify-cli