BootstrapDetectionError: Unable to detect CDK bootstrap stack due to permission issues. On backend build Unable to deploy monorepo.

crafael23 commented 1 week ago

Environment information

System:
  OS: macOS 15.1
  CPU: (11) arm64 Apple M3 Pro
  Memory: 111.02 MB / 18.00 GB
  Shell: /bin/zsh
Binaries:
  Node: 22.11.0 - ~/Library/pnpm/node
  Yarn: undefined - undefined
  npm: 10.9.0 - ~/Library/pnpm/npm
  pnpm: 9.12.3 - ~/Library/pnpm/pnpm
NPM Packages:
  @aws-amplify/auth-construct: Not Found
  @aws-amplify/backend: 1.7.0
  @aws-amplify/backend-auth: Not Found
  @aws-amplify/backend-cli: 1.4.2
  @aws-amplify/backend-data: Not Found
  @aws-amplify/backend-deployer: Not Found
  @aws-amplify/backend-function: Not Found
  @aws-amplify/backend-output-schemas: Not Found
  @aws-amplify/backend-output-storage: Not Found
  @aws-amplify/backend-secret: Not Found
  @aws-amplify/backend-storage: Not Found
  @aws-amplify/cli-core: Not Found
  @aws-amplify/client-config: Not Found
  @aws-amplify/deployed-backend-client: Not Found
  @aws-amplify/form-generator: Not Found
  @aws-amplify/model-generator: Not Found
  @aws-amplify/platform-core: Not Found
  @aws-amplify/plugin-types: Not Found
  @aws-amplify/sandbox: Not Found
  @aws-amplify/schema-generator: Not Found
  aws-amplify: 6.8.0
  aws-cdk: 2.166.0
  aws-cdk-lib: 2.166.0
  typescript: 5.5.4
AWS environment variables:
  AWS_STS_REGIONAL_ENDPOINTS = regional
  AWS_NODEJS_CONNECTION_REUSE_ENABLED = 1
  AWS_SDK_LOAD_CONFIG = 1
No CDK environment variables

Describe the bug

I've been trying to setup a custom repository for a monorepo that will have several apps hosted on it.

The objective is to be able to share the backend with multple web and mobile apps.

I cant seem to be able to build the backend.

I get the following error within the console when attempting to build it.

BUILD.txt

this is the current repo

https://github.com/KinuQ/kinuqmono

Im honestly not sure what is it that I'm doing wrong.

I had another single repo app with a full fledged backend, and it is able to just go ahead and get deployed without an issue.

this is my amplify.yml for the CI process

version: 1
backend:
  phases:
    preBuild:
      commands:
        - corepack enable
        - pnpm install
    build:
      commands:
        - echo $AWS_BRANCH 
        - echo $AWS_APP_ID
        - pwd
        - cd packages/amplify
        - pwd
        - pnpm exec ampx pipeline-deploy --branch $AWS_BRANCH --app-id $AWS_APP_ID
        - cd ...

frontend:
  phases:
    build:
      commands:
        - pnpm turbo build
  artifacts:
    baseDirectory: apps/placeholderapp/dist
    files:
      - "**/*"
  cache:
    paths:
      - node_modules/**/*

Reproduction steps

fork my repo,

attempt to deploy on aws.

ykethan commented 1 week ago

Hey @crafael23, thank you for reaching out. From the build logs the message

credentials could not be used to assume 'arn:aws:iam::<account-id>:role/cdk-hnb659fds-deploy-role-<account-id>-us-east-2', but are for the right account. Proceeding anyway.

indicating the role may have been deleted or this may also occur if the account has Service Control Policies that could be enforced on the IAM user/role on an account level within the organization.

Does local sandbox throw a similar error message? On the IAM console, could you confirm if the role does exist? additionally, confirm if the account does not have any service control policies?

crafael23 commented 1 week ago

@ykethan , for the moment local sandbox doesn't produce the same error

I do have other proyects hosted, a nextjs app and a little react SPA. Its only this one that gives the the problem, if I remove backend deployment completely from te amplify.yml the problem goes away.

crafael23 commented 1 week ago

@ykethan btw, because this is a test that I'm doing, I'm doing this on my personal root account. I've not touched Service control Policies.

crafael23 commented 1 week ago

Hey @crafael23, thank you for reaching out. From the build logs the message
credentials could not be used to assume 'arn:aws:iam::<account-id>:role/cdk-hnb659fds-deploy-role-<account-id>-us-east-2', but are for the right account. Proceeding anyway.�
indicating the role may have been deleted or this may also occur if the account has Service Control Policies that could be enforced on the IAM user/role on an account level within the organization.

Does local sandbox throw a similar error message? On the IAM console, could you confirm if the role does exist? additionally, confirm if the account does not have any service control policies?

I just verified and the role does exist.

ykethan commented 1 week ago

Hey @crafael23, thank you for the information. For a quick confirmation, does the app have a service role attached in the general settings?

as i did run into this error message when the service role was not present, on attaching the role the deployment was successfull

crafael23 commented 1 week ago

@ykethan , the field for the service role is empty for this app. Comparing it to the other react spa that i have there is no differnece, but comparing it to another nextjs app that i have deployed it appears to only have AmplifySSRLoggingRole. Ill try adding the deploy role.

crafael23 commented 1 week ago

I guess thats not what im supposed to do

ykethan commented 1 week ago

@crafael23 does the service role attached have AmplifyBackendDeployFullAccess attached to it?

i was able to deploy the KinuQ/kinuqmono app using the following yml and the service role attached

version: 1
applications:
  - appRoot: .
    backend:
      phases:
        preBuild:
          commands:
            - npm install -g pnpm
            - corepack enable
            - pnpm install
        build:
          commands:
            - echo $AWS_BRANCH
            - echo $AWS_APP_ID
            - cd packages/amplify
            - pnpm exec ampx pipeline-deploy --branch $AWS_BRANCH --app-id $AWS_APP_ID
            - cd ../..
    frontend:
      phases:
        build:
          commands:
            - pnpm turbo run build --filter="./apps/placeholderapp"
      artifacts:
        baseDirectory: apps/placeholderapp/dist
        files:
          - "**/*"
      cache:
        paths:
          - node_modules/**/*
          - .pnpm-store/**/*

crafael23 commented 1 week ago

@ykethan CDK deploy role doesnt have the amplify AmplifyBackendDeployFullAccess

It just has a default role with this description

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "CDKPreDeploy",
            "Effect": "Allow",
            "Action": [
                "cloudformation:DescribeStacks",
                "cloudformation:DescribeStackEvents",
                "cloudformation:GetTemplate",
                "cloudformation:ListStackResources",
                "cloudformation:GetTemplateSummary",
                "cloudformation:DeleteStack"
            ],
            "Resource": [
                "arn:aws:cloudformation:*:*:stack/amplify-*",
                "arn:aws:cloudformation:*:*:stack/CDKToolkit/*"
            ],
            "Condition": {
                "StringEquals": {
                    "aws:ResourceAccount": "${aws:PrincipalAccount}"
                }
            }
        },
        {
            "Sid": "AmplifyMetadata",
            "Effect": "Allow",
            "Action": [
                "amplify:ListApps",
                "cloudformation:ListStacks",
                "ssm:DescribeParameters",
                "appsync:GetIntrospectionSchema",
                "amplify:GetBackendEnvironment"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Sid": "AmplifyHotSwappableResources",
            "Effect": "Allow",
            "Action": [
                "appsync:GetSchemaCreationStatus",
                "appsync:StartSchemaCreation",
                "appsync:UpdateResolver",
                "appsync:ListFunctions",
                "appsync:UpdateFunction",
                "appsync:UpdateApiKey"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Sid": "AmplifyHotSwappableFunctionResource",
            "Effect": "Allow",
            "Action": [
                "lambda:InvokeFunction",
                "lambda:UpdateFunctionCode",
                "lambda:GetFunction",
                "lambda:UpdateFunctionConfiguration"
            ],
            "Resource": [
                "arn:aws:lambda:*:*:function:amplify-*"
            ],
            "Condition": {
                "StringEquals": {
                    "aws:ResourceAccount": "${aws:PrincipalAccount}"
                }
            }
        },
        {
            "Sid": "AmplifySandboxLambdaLogsStreamingListTags",
            "Effect": "Allow",
            "Action": [
                "lambda:ListTags"
            ],
            "Resource": [
                "arn:aws:lambda:*:*:function:amplify-*"
            ]
        },
        {
            "Sid": "AmplifySandboxLambdaLogsStreamingFilterLogEvents",
            "Effect": "Allow",
            "Action": [
                "logs:FilterLogEvents"
            ],
            "Resource": [
                "arn:aws:logs:*:*:log-group:/aws/lambda/amplify-*:*"
            ]
        },
        {
            "Sid": "AmplifySchema",
            "Effect": "Allow",
            "Action": [
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::*amplify*",
                "arn:aws:s3:::cdk-*-assets-*-*"
            ],
            "Condition": {
                "StringEquals": {
                    "aws:ResourceAccount": "${aws:PrincipalAccount}"
                }
            }
        },
        {
            "Sid": "CDKDeploy",
            "Effect": "Allow",
            "Action": [
                "sts:AssumeRole"
            ],
            "Resource": [
                "arn:aws:iam::*:role/cdk-*-deploy-role-*-*",
                "arn:aws:iam::*:role/cdk-*-file-publishing-role-*-*",
                "arn:aws:iam::*:role/cdk-*-image-publishing-role-*-*",
                "arn:aws:iam::*:role/cdk-*-lookup-role-*-*"
            ],
            "Condition": {
                "StringEquals": {
                    "aws:ResourceAccount": "${aws:PrincipalAccount}"
                }
            }
        },
        {
            "Sid": "AmplifySSM",
            "Effect": "Allow",
            "Action": [
                "ssm:GetParametersByPath",
                "ssm:GetParameters",
                "ssm:GetParameter"
            ],
            "Resource": [
                "arn:aws:ssm:*:*:parameter/amplify/*",
                "arn:aws:ssm:*:*:parameter/cdk-bootstrap/*"
            ],
            "Condition": {
                "StringEquals": {
                    "aws:ResourceAccount": "${aws:PrincipalAccount}"
                }
            }
        },
        {
            "Sid": "AmplifyModifySSMParam",
            "Effect": "Allow",
            "Action": [
                "ssm:PutParameter",
                "ssm:DeleteParameter",
                "ssm:DeleteParameters"
            ],
            "Resource": "arn:aws:ssm:*:*:parameter/amplify/*",
            "Condition": {
                "StringEquals": {
                    "aws:ResourceAccount": "${aws:PrincipalAccount}"
                }
            }
        },
        {
            "Sid": "AmplifyDiscoverRDSVpcConfig",
            "Effect": "Allow",
            "Action": [
                "rds:DescribeDBProxies",
                "rds:DescribeDBInstances",
                "rds:DescribeDBClusters",
                "ec2:DescribeSubnets",
                "rds:DescribeDBSubnetGroups"
            ],
            "Resource": [
                "arn:aws:rds:*:*:db:*",
                "arn:aws:rds:*:*:cluster:*",
                "arn:aws:rds:*:*:db-proxy:*",
                "arn:aws:rds:*:*:subgrp:*",
                "arn:aws:ec2:*:*:subnet/*"
            ],
            "Condition": {
                "StringEquals": {
                    "aws:ResourceAccount": "${aws:PrincipalAccount}"
                }
            }
        }
    ]
}

Ill try it out in a second.

crafael23 commented 1 week ago

This is happening now @ykethan

crafael23 commented 1 week ago

@ykethan

I'm unsure weather this are the right settings for this

Since I see that in the yaml you provided has the approot on ./apps.

ykethan commented 1 week ago

@crafael23 i was able to deploy this successfully using the following. could you try this and let us know if it deploys

version: 1
applications:
  - appRoot: .
    backend:
      phases:
        preBuild:
          commands:
            - npm install -g pnpm
            - corepack enable
            - pnpm install
        build:
          commands:
            - echo $AWS_BRANCH
            - echo $AWS_APP_ID
            - cd packages/amplify
            - pnpm exec ampx pipeline-deploy --branch $AWS_BRANCH --app-id $AWS_APP_ID
            - cd ../..
    frontend:
      phases:
        build:
          commands:
            - pnpm turbo run build --filter="./apps/placeholderapp"
      artifacts:
        baseDirectory: apps/placeholderapp/dist
        files:
          - "**/*"
      cache:
        paths:
          - node_modules/**/*
          - .pnpm-store/**/*

crafael23 commented 1 week ago

@ykethan I created a new Role and attached the AmplifyBackendDeployFullAccess policy to it, assigned it to the app and worked properly.

Specific steps i took

Just to make it clear, I'm using Turborepo. But it should work almost the same for any other monorepo solution.

here is my project's structure.

.
├── README.md
├── amplify.yaml
├── apps
│   └── placeholderapp
│       ├── README.md
│       ├── dist
│       │   ├── assets
│       │   │   ├── index-CFbQLCJg.js
│       │   │   └── index-n_ryQ3BS.css
│       │   ├── index.html
│       │   └── vite.svg
│       ├── eslint.config.js
│       ├── index.html
│       ├── package.json
│       ├── public
│       │   └── vite.svg
│       ├── src
│       │   ├── App.css
│       │   ├── App.tsx
│       │   ├── assets
│       │   │   └── react.svg
│       │   ├── index.css
│       │   ├── main.tsx
│       │   └── vite-env.d.ts
│       ├── tsconfig.app.json
│       ├── tsconfig.json
│       ├── tsconfig.node.json
│       └── vite.config.ts
├── package.json
├── packages
│   ├── amplify
│   │   ├── amplify
│   │   │   ├── auth
│   │   │   │   └── resource.ts
│   │   │   ├── backend.ts
│   │   │   ├── data
│   │   │   │   └── resource.ts
│   │   │   ├── package.json
│   │   │   └── tsconfig.json
│   │   ├── amplify_outputs.json
│   │   ├── index.ts
│   │   ├── output.ts
│   │   └── package.json
│   ├── eslint-config
│   │   ├── README.md
│   │   ├── library.js
│   │   ├── next.js
│   │   ├── package.json
│   │   └── react-internal.js
│   ├── typescript-config
│   │   ├── base.json
│   │   ├── nextjs.json
│   │   ├── package.json
│   │   └── react-library.json
│   └── ui
│       ├── package.json
│       ├── src
│       │   ├── button.tsx
│       │   ├── card.tsx
│       │   └── code.tsx
│       ├── tsconfig.json
│       ├── tsconfig.lint.json
│       └── turbo
│           └── generators
│               ├── config.ts
│               └── templates
│                   └── component.hbs
├── pnpm-lock.yaml
├── pnpm-workspace.yaml
└── turbo.json

20 directories, 51 files

Add the repository and branch

Specify that the app is a monorepo and that the root directory to be . (the root off the whole monorepo)

2 Edit the build yml file

version: 1
applications:
  - appRoot: .
    backend:
      phases:
        preBuild:
          commands:
            # Install pnpm globally
            - npm install -g pnpm
            # Enable corepack to manage package managers
            - corepack enable
            # Install dependencies using pnpm
            - pnpm install
        build:
          commands:
            # Print the AWS branch name
            - echo $AWS_BRANCH
            # Print the AWS app ID
            - echo $AWS_APP_ID
            # Change directory to the amplify package
            - cd packages/amplify
            # Execute the amplify pipeline deploy command with the branch and app ID
            - pnpm exec ampx pipeline-deploy --branch $AWS_BRANCH --app-id $AWS_APP_ID
            # Change back to the root directory
            - cd ../..
    frontend:
      phases:
        build:
          commands:
            # Run the build command for the placeholderapp using turbo
            - pnpm turbo run build --filter="./apps/placeholderapp"
      artifacts:
        baseDirectory: apps/placeholderapp/dist
        files:
          # Include all files in the dist directory
          - "**/*"
      cache:
        paths:
          # Cache node_modules to speed up subsequent builds
          - node_modules/**/*
          # Cache pnpm store to speed up subsequent builds
          - .pnpm-store/**/*

Create a custom IAM role After that I created a role for the sole purpose of allowing the app to deploy cdk resources or amplify resources.

Both AdministratorAccess-Amplify and AmplifyBackendDeployFullAccess are good for this purpose.

Save and deploy the app and it will start deploying but it will fail since it doesn't have the right role to deploy backend resources.

Modify the Service Role for it to be able to deploy the backend resources.

Add the custom role that we created previously and redeploy

It should now be able to deploy without an issue.

Thank you for providing more insight on how to fix it and why was it that it was failing @ykethan.

aws-amplify / amplify-backend