aws-amplify / amplify-hosting

AWS Amplify Hosting provides a Git-based workflow for deploying and hosting fullstack serverless web applications.
https://aws.amazon.com/amplify/hosting/
Apache License 2.0
461 stars 116 forks source link

IAM permission with NextJS #3205

Open philschmid opened 1 year ago

philschmid commented 1 year ago

Before opening, please confirm:

App Id

-

AWS Region

us-east-1

Amplify Hosting feature

SSR

Question

Hello,

I have a question regarding IAM permissions. I successfully deployed NextJS 13 using amplify. Now, I want to use AWS service via the aws javascript SDK in my api/ functions. How can i provide credentials in a secure way to it? I know that i could create a user and pass the accessKeyId and secretAccessKey as env, but i would like to avoid this. How can I assign a role to my NextJs app?

philschmid commented 1 year ago

cc @hloriana

calavera commented 1 year ago

You can change the role for your app in the settings:

image

Keep in mind that we need so basic policy permissions to deliver logs to CloudWatch, you can see them here:

image

philschmid commented 1 year ago

@calavera the service role is the one used in the edge lambda functions? Do i need to add the permissions mentioned here: https://github.com/aws-amplify/amplify-hosting/blob/main/FAQ.md#error-accessdenied-access-denied as well?

calavera commented 1 year ago

You don't need those permissions anymore if you're deploying Next 13. That documentation is outdated, unfortunately. We need to update that.

philschmid commented 1 year ago

Thank you for letting me know! I assume those are not longer needed due to the "Trust relationship" which is added?

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "amplify.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

Correct? so the "custom" role needs those

calavera commented 1 year ago

correct. The custom role needs that trust relationship.

philschmid commented 1 year ago

No success yet. I created an IAM role with CDK

   // create iam role for amplify
    const role = new iam.Role(this, 'AmplifyRole', {
      assumedBy: new iam.ServicePrincipal('amplify.amazonaws.com'),
    });
    // add permissions to write logs
    role.addToPolicy(new iam.PolicyStatement({
      actions: ['logs:CreateLogStream','logs:CreateLogGroup','logs:DescribeLogGroups','logs:PutLogEvents'],
      resources: ['*'],
    }));
    // add permissions to create users in table
    table.grantReadWriteData(role)

which gets successfully created and and permissions seems to be correct. But then inside my Amplify NextJS app I get the following error

Could not load credentials from any providers {
--
message: 'Could not load credentials from any providers',
stack: 'CredentialsProviderError: Could not load credentials from any providers\n' +
'    at /var/task/node_modules/@aws-sdk/credential-provider-node/dist-cjs/defaultProvider.js:13:11\n' +
'    at /var/task/node_modules/@aws-sdk/property-provider/dist-cjs/chain.js:11:28\n' +
'    at processTicksAndRejections (node:internal/process/task_queues:96:5)\n' +
'    at async coalesceProvider (/var/task/node_modules/@aws-sdk/property-provider/dist-cjs/memoize.js:14:24)\n' +
'    at async SignatureV4.credentialProvider (/var/task/node_modules/@aws-sdk/property-provider/dist-cjs/memoize.js:33:24)\n' +
'    at async SignatureV4.signRequest (/var/task/node_modules/@aws-sdk/signature-v4/dist-cjs/SignatureV4.js:86:29)\n' +
'    at async /var/task/node_modules/@aws-sdk/middleware-signing/dist-cjs/middleware.js:16:18\n' +
'    at async /var/task/node_modules/@aws-sdk/middleware-retry/dist-cjs/retryMiddleware.js:27:46\n' +
'    at async /var/task/node_modules/@aws-sdk/middleware-logger/dist-cjs/loggerMiddleware.js:5:22\n' +
'    at async getUserByAccount (/var/task/node_modules/@next-auth/dynamodb-adapter/dist/index.js:59:26)',
name: 'CredentialsProviderError'
}

I am trying to use the @aws-sdk/lib-dynamodb in an api/ function. with

const config: DynamoDBClientConfig = {
  region: process.env.NEXT_AUTH_AWS_REGION,
};

const client = DynamoDBDocument.from(new DynamoDB(config), {
  marshallOptions: {
    convertEmptyValues: true,
    removeUndefinedValues: true,
    convertClassInstanceToMap: true,
  },
})

the repository is public as well: https://github.com/philschmid/aws-marketplace-example/blob/main/app/pages/api/auth/%5B...nextauth%5D.ts#L8

calavera commented 1 year ago

I've been reviewing this with the team, and it might actually not work, and I was wrong 🤦 The role that you setup in our Console is not the same role that the function gets execution credentials from. So the permissions won't be propagated as expected.

So the only solution for now is to inject the credentials in the environment. We're going to look if we can prioritize this work in the near future, but we cannot make any commitments at the moment.

philschmid commented 1 year ago

Thank you for the response! Too bad it is not working. Adding secrets as an environment doesn't sound super secure or something you should do. I hope you ll find time to add it.

williamrjribeiro commented 1 year ago

@philschmid may be a dumb question but, did you import the DynamoDB Table with amplify import storage command and afterward amplify push?

Or did you create a new DynamoDB Table with amplify add storage?

philschmid commented 1 year ago

@williamrjribeiro i am not using the amplify cli or amplify functions. I am using CDK/Cloudformation to create everything and then only have a "NextJS" app, where the NextJS functions/api are -> amplify hosting.

nstankov-bg commented 1 year ago

I shall +1 this.

To the dot, the exact same problem is happening, forcing the team to potentially "hardcode" credentials.

smilhas commented 1 year ago

+1 Thank you!

eezdev commented 1 year ago

+1 any news about this? This is very important to secure ssr apps using iam

GabrieleMazzola commented 1 year ago

Same problem here, is there a plan to fix this by any chance?

bajcmartinez commented 1 year ago

+1, seems a pretty basic thing to have.

Lagyu commented 1 year ago

I know this is not the best solution, but we can generate .env file with IAM user credential in Build process as workaround:

# environment variables config
DUMMY_PREFIX_AWS_ACCESS_KEY_ID = YourAccessKey
DUMMY_PREFIX_AWS_SECRET_ACCESS_KEY = YourSecret
        build:
          commands:
            - env | sed "s/DUMMY_PREFIX_//g" >> .env.production
            - yarn run build
AkihiroTakamura commented 1 year ago

+1 Since Amplify Hosting can manage multiple stages, we are hoping to implement an IAM Role that can be assigned to each stage.

Currently, as mentioned in the above comment, we cannot find a way other than embedding the access key and secret access key in the environment variables and build of amplify. but this would be difficult to use, especially in a production. because it is a persistent authentication information.

HA55EHH commented 1 year ago

+1 currently using .env.production workaround…

Neo-Ciber94 commented 1 year ago

I know this is not the best solution, but we can generate a .env file with IAM user credential in Build process as workaround:

# environment variables config
DUMMY_PREFIX_AWS_ACCESS_KEY_ID = YourAccessKey
DUMMY_PREFIX_AWS_SECRET_ACCESS_KEY = YourSecret
        build:
          commands:
            - env | sed "s/DUMMY_PREFIX_//g" >> .env.production
            - yarn run build

This works, but would be cool to not expose my credentials in the Amplify console

dannyk08 commented 1 year ago

Having the same issues here as well Really don't like the workaround 😕

KylerD commented 1 year ago

Any update on this?

sfedorov-at-wiley commented 1 year ago

Oh my. Finally found this We have exactly the same issue Using DynamoDB to render some SSR content and @aws-sdk/client-dynamodb does not see the service role past build stage. Our SRE team will not allow us to set credentials in the environment variables for sure. Is there any workaround?

philschmid commented 1 year ago

@sfedorov-at-wiley i switched to sst. https://docs.sst.dev

There you have full control since you will create a real lambda function which is not abstracted.

MaxiMittel commented 1 year ago

Or if you don’t like SST you can also use CDK with OpenNext https://github.com/jetbridge/cdk-nextjs

KevinNha commented 1 year ago

Following this thread. I guess the only workaround is to use other hosting besides Amplify or set the env variable?

yb1727 commented 1 year ago

I had the same issue where credentials were stored as env variables. The way I worked around the issue is to store the credentials in AWS secrets manager, and creating an env variable which just holds the secret id . The secret access policy was set to allow 'read' access only to the amplify pipeline service role, so that no one else can see the credentials. During the build, the amplify pipeline fetches those credentials and adds them to .env

antondelpiero commented 1 year ago

would be great not to create an IAM user just to have for the Access and Secret Key. is there any updates on this? Thanks is advance

LightandSound commented 1 year ago

any update on this?

kaito-hao commented 1 year ago

Spent days and found the root cause here. Please share some progress or let us know how to vote to accelerate this feature request. Thank you.

Gilaad-Elstein commented 11 months ago

Not sure where to support this, so i'll say here we need this too. Thank you.

nkeysinstil commented 11 months ago

We are also experiencing this issue currently! Any update on this?

endgameviable commented 11 months ago

Will add my name to the chorus. This is the exact issue I'm experiencing. Passing token and secret through environment variables so that Next.js api route handlers can access aws resources, wanted to find a way to avoid doing that. Looking forward to updates.

jordanpurinton commented 11 months ago

Looking for this functionality as well

AndreasCaldewei commented 11 months ago

I'm also interested in this feature.

glucn commented 10 months ago

Any update? Thx!

AllenZ05 commented 10 months ago

So, is there any update? We kinda need it...

swaminator commented 10 months ago

We published a blog post to describe accessing resources server side: https://aws.amazon.com/blogs/mobile/accessing-resources-in-a-amazon-virtual-private-cloud-amazon-vpc-from-next-js-api-routes/

kaito-hao commented 10 months ago

This is a similar solution that pass the secret key and id to ENV, which already discussed above.

moesmufti commented 9 months ago

Also + 1 still having this issue. I have a role assigned to the amplify console that has full dynamodb access, but getting the CredentialsProviderError: Could not load credentials from any providers during runtime in production. Local works using the credentials file.

github-actions[bot] commented 9 months ago

This has been identified as a feature request. If this feature is important to you, we strongly encourage you to give a 👍 reaction on the request. This helps us prioritize new features most important to you. Thank you!

callumthomson commented 5 months ago

I've been trying to solve this exact same issue for quite some time now and have landed here. From what I have read, it seems like there is no proper way to grant IAM permissions to the server side of my NextJS app hosted in Amplify. This is definitely something that is absolutely crucial to have.

webdesignbystephen commented 4 months ago

July 2024: Still annoying developers!

triwonderdigital commented 3 months ago

From the docs here: https://docs.aws.amazon.com/amplify/latest/userguide/ssr-environment-variables.html I have these set:

`version: 1 frontend: phases: preBuild: commands:

After checking the logs we see: CredentialsProviderError: Could not load credentials from any providers This is the same issue as part of this ticket? OK after spending 3 days going round in circles trying to get our app running with server api. I decided to try Vercel up and running with no errors in 10 minutes, bye bye amplify...

hans25041 commented 3 months ago

This is critical. We've been trying to transition away from persistent credentials and IAM users for a long time. It's surprising to me that a modern, serverless Amazon service would encourage (require?) the use of long term credentials. There should be an execution role that the NextJS app uses when it runs. I'm too invested to move my current project away from Amplify, but I won't recommend that my company uses Amplify again until this problem is solved. I know AWS likes to say that "security is job #1." That means that this should be a top priority.

HoltSpalding commented 2 months ago

Bump

Pito1992 commented 2 months ago

Hello guys,

I faced a similar situation and here was my solution. The solution suits my case but I think I may fit your side or anybody else, and I'm happy if it can help. I was a person like you who was looking for a solution in StackOverflow; GitHub issues; I tried to modify the AWS IAM policy, role... but there was no luck. Then, I was reading back to the AWS Amplify documents carefully and this link below was the savior. It took me 2 days to recognize this one. Honestly, AWS Amplify documents are not clear enough and its search results are stupid. https://docs.amplify.aws/react/build-a-backend/add-aws-services/geo/amazon-location-sdk/

Regarding the link, it requires you to attach credentials to any AWS-SDK you want to use rather than using the local AWS config. For example, I was using "@aws-sdk/client-iot" and it should be like this

const client = new IoTClient({
    credentials,
    region: env.REGION
 })

In my case, a user should log in to the app and use the services inside so I don't need to set my AWS config by env or secret params. Just create a user pool and manage policies on it.

gennaroanesi commented 2 months ago

I was open to injecting the secrets as env variables using the ssm cli during build, but that didn't seem to work as well (secrets json is empty for me even though I have them set up in the console)

Gilaad-Elstein commented 2 months ago

I was open to injecting the secrets as env variables using the ssm cli during build, but that didn't seem to work as well (secrets json is empty for me even though I have them set up in the console)

Have you tried this approach? Until we get IAM support this is the best way I found with Amplify and NextJS. It is a deal breaker for some projects for sure, but it does work. Just mind what you expose on server and client side, on the console etc.

mauerbac commented 1 month ago

Hello - this issue has now been moved to in-development! We are looking at a solution where you would create an IAM role and then have the ability to attach that role within the Amplify Hosting console.

I'm curious what types of roles/policies folks want to attach? What is the use case? What will the policies do?

Thank you!