support usage of CDK modules in overrides, receiving `Skipping override due to VMError`

mtliendo commented 2 years ago

Before opening, please confirm:

[X] I have installed the latest version of the Amplify CLI (see above), and confirmed that the issue still persists.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
[X] I have removed any sensitive information from my code snippets and submission.

How did you install the Amplify CLI?

npm

If applicable, what version of Node.js are you using?

16.13.0

Amplify CLI Version

7.6.26

What operating system are you using?

mac

Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.

No manual changes made

Amplify Categories

storage

Amplify Commands

override

Describe the bug

After configuring an S3 bucket with amplify add storage and following the prompts, I wanted to apply a policy to my bucket so that anyone can view the uploaded images--not just unauthenticated users.

I ran amplify override storage and created a bucket instance (since Amplify generates a CfnBucket) and attached a policy to it.

import { AmplifyS3ResourceTemplate } from '@aws-amplify/cli-extensibility-helper'
import * as s3 from '@aws-cdk/aws-s3'
import * as iam from '@aws-cdk/aws-iam'

export function override(resources: AmplifyS3ResourceTemplate) {
    const productsBucket = s3.Bucket.fromBucketArn(
        this,
        'productsBucket',
        resources.s3Bucket.attrArn
    )

    productsBucket.addToResourcePolicy(
        new iam.PolicyStatement({
            actions: ['s3:Get*', 's3:List*'],
            resources: [`${resources.s3Bucket.attrArn}/public/productimages/**/*`],
            principals: [new iam.AnyPrincipal()],
        })
    )
}

Note that I did not manually import the modules at the top as they are already installed by Amplify.

Upon running amplify push -y I received the following error

Expected behavior

I would expect the command to push up my resources and the s3 console to reflect the newly added policy.

If I manually add the policy in the console it works but I don't want to introduce drift (I deleted it afterward).

Reproduction steps

amplify add storage
select read access for any options
amplify override storage
add the following statement

import { AmplifyS3ResourceTemplate } from '@aws-amplify/cli-extensibility-helper'
import * as s3 from '@aws-cdk/aws-s3'
import * as iam from '@aws-cdk/aws-iam'

export function override(resources: AmplifyS3ResourceTemplate) {
    const productsBucket = s3.Bucket.fromBucketArn(
        this,
        'productsBucket',
        resources.s3Bucket.attrArn
    )

    productsBucket.addToResourcePolicy(
        new iam.PolicyStatement({
            actions: ['s3:Get*', 's3:List*'],
            resources: [`${resources.s3Bucket.attrArn}/public/productimages/**/*`],
            principals: [new iam.AnyPrincipal()],
        })
    )
}

amplify push -y

GraphQL schema(s)

```graphql # Put schemas below this line ```

Log output

``` # Put your logs below this line Fetching updates to backend environment: dev from the cloud.🛑 Error: Skipping override due to VMError: Cannot find mo ✔ Successfully pulled backend environment dev from the cloud. ⠸ Building resource storage/productsBucket🛑 Error: Skipping override due to VMError: Cannot find module 'os' ```

Additional information

The option to have public read (for anyone) access should be an option when adding storage.

use case: I am creating a storefront and stripe keeps a list of product URLs so that when a customer checks out it can display the product on their hosted checkout form.

josefaidt commented 2 years ago

Hey @mtliendo :wave: thanks for raising this! It appears you are encountering the issue described in this feature request https://github.com/aws-amplify/amplify-cli/issues/9063 where we are unable to import other npm packages (like @aws-sdk/*) or files inside an override. This is due to the overrides' sandbox execution environment, and is something the team is looking to improve 🙂

As a workaround, we can manually apply the IAM policy object instead of using the IAM module to construct it. Example from this comment on a related issue

resources.s3AuthPublicPolicy.policyDocument.Statement[0] = Object.assign(
    resources.s3AuthPublicPolicy.policyDocument.Statement[0], 
    { Action: [ "s3:PutObject","s3:PutObjectAcl","s3:GetObject"] }
)

Upon running amplify build we can inspect the CloudFormation template in the build/ directory of our storage resource and observe the policies.

Marking as a duplicate feature request

mtliendo commented 2 years ago

This doesn't seem to work in my use case. I'm trying to add a resource policy and need to specify the Principal. The above solution seems to only apply to identity policies.

mtliendo commented 2 years ago

If I attach the policy in the console, it works just fine. But I was hoping for a solution that would codify it with amplify override/custom.

tillenium commented 2 years ago

I see multiple issues open pointing to the same problem, open for months. Why is Amplify team not looking into this? Not able to use aws-cdk libraries render a lot of overriding useless, for example enabling lambda trigger for cognito. This stabs bias for action and customer obsession right at heart!!

chrisl777 commented 1 year ago

I'm having a similar issue, trying to override S3 Storage:

const statement = {
    Sid: "PublicReadGetObject",
    Effect: "Allow",
    Principal: "*",
    Action: ["s3:GetObject"],
    Resources: `${resources.s3Bucket.attrArn}/public/*`
  }

  resources.s3GuestReadPolicy.policyDocument.Statement[0] = Object.assign(
    resources.s3GuestReadPolicy.policyDocument.Statement[0],
    statement 
  )

Error output:

🛑 InvalidOverrideError: Executing overrides failed.
    at AmplifyS3ResourceStackTransform.applyOverrides (/snapshot/amplify-cli/build/node_modules/@aws-amplify/amplify-category-storage/lib/provider-utils/awscloudformation/cdk-stack-builder/s3-stack-transform.js:130:17)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async AmplifyS3ResourceStackTransform.transform (/snapshot/amplify-cli/build/node_modules/@aws-amplify/amplify-category-storage/lib/provider-utils/awscloudformation/cdk-stack-builder/s3-stack-transform.js:220:5)
    at async transformS3ResourceStack (/snapshot/amplify-cli/build/node_modules/@aws-amplify/amplify-category-storage/lib/provider-utils/awscloudformation/cdk-stack-builder/s3-stack-transform.js:50:5)
    at async transformCategoryStack (/snapshot/amplify-cli/build/node_modules/@aws-amplify/amplify-category-storage/lib/index.js:195:5)
    at async transformResourceWithOverrides (/snapshot/amplify-cli/build/node_modules/@aws-amplify/amplify-provider-awscloudformation/lib/override-manager/transform-resource.js:66:9)
    at async Object.run (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/commands/build.js:27:13)
    at async Object.executeAmplifyCommand (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/index.js:194:9)
    at async executePluginModuleCommand (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/execution-manager.js:139:5)
    at async executeCommand (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/execution-manager.js:37:9)
    at async Object.run (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/index.js:121:5)
🛑 There was an error building the resource

aws-amplify / amplify-cli