Lambda Function GraphQL Authentication issues

Lorenzohidalgo commented 2 years ago

Before opening, please confirm:

[X] I have installed the latest version of the Amplify CLI (see above), and confirmed that the issue still persists.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
[X] I have removed any sensitive information from my code snippets and submission.

How did you install the Amplify CLI?

No response

If applicable, what version of Node.js are you using?

No response

Amplify CLI Version

7.6.26

What operating system are you using?

Windows (amplify cli on WSL)

Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.

Auth:

Custom verfication code email template

DynamoDB:

Added GSIs + Auto BackUp Options

Amplify Categories

function, api

Amplify Commands

add

Describe the bug

I've created a new function using the latest version of the CLI and granted access to the API resources as stated in the docs.

The generated function resources seem to miss Authorization to access AppSync schema, since any type of access attempt ends up in the following response:

{
   "data":{
      "getUsers":null
   },
   "errors":[
      {
         "path":[
            "getUsers"
         ],
         "data":null,
         "errorType":"Unauthorized",
         "errorInfo":null,
         "locations":[
            {
               "line":2,
               "column":9,
               "sourceName":null
            }
         ],
         "message":"Not Authorized to access getUsers on type Query"
      }
   ]
}

I've double-checked and the CLI recognizes/shows that the generated function has the required Resource access permissions. Existing functions created following the same exact steps (with previous CLI versions) are working correctly.

Expected behavior

The function should have access to the resources we specified during the creation process.

Reproduction steps

amplify add function
follow all required steps & grant access to the AppSync API
try to retrieve data by accessing Appsync

Used code to generate calls to Appsync:

const aws = require('aws-sdk');
const https = require('https');
const urlParse = require("url").URL;
const appsyncUrl = process.env.API_<PROJECTNAME>_GRAPHQLAPIENDPOINTOUTPUT;
const region = process.env.REGION;
const endpoint = new urlParse(appsyncUrl).hostname.toString();
const apiKey = process.env.API_<PROJECTNAME>_GRAPHQLAPIKEYOUTPUT;
const graphqlQuerys = require('./querys.js');

async function graphQLQuery(query, variables){
    const req = new aws.HttpRequest(appsyncUrl, region);

    req.method = "POST";
    req.path = "/graphql";
    req.headers.host = endpoint;
    req.headers["Content-Type"] = "application/json";
    if(variables != null){
        req.body = JSON.stringify({
            query: query,
            variables: variables
        });
    } else {
        req.body = JSON.stringify({
            query: query
        });
    }

    if (apiKey) {
        req.headers["x-api-key"] = apiKey;
    } else {
        const signer = new AWS.Signers.V4(req, "appsync", true);
        signer.addAuthorization(AWS.config.credentials, AWS.util.date.getDate());
    }

    const data = await new Promise((resolve, reject) => {
        const httpRequest = https.request({ ...req, host: endpoint }, (result) => {
            let data = "";

            result.on("data", (chunk) => {
                data += chunk;
            });

            result.on("end", () => {
                resolve(JSON.parse(data.toString()));
            });
        });

        httpRequest.write(req.body);
        httpRequest.end();
    });
    return data;
}

GraphQL schema(s)

```graphql # Put schemas below this line type Users @model @auth(rules: [{allow: private}]) { id: ID! pid: ID email: AWSEmail! @index(name: "usersByEmail", queryField: "usersByEmail") name: String cognito_uid: ID avatarKey: String phoneNumber: String joinDate: AWSDateTime! lastLogIn: AWSDateTime description: String lastSpacePID: ID pushIdentifiers: [String] pushNotifications: Boolean emailNotifications: Boolean language: String isPartner: Boolean isActive: Boolean } ```

Log output

``` # Put your logs below this line ```

Additional information

functions created with previous cli versions following the same steps still work properly. Some of them where created before the GraphQl migration.

anttironty commented 2 years ago

I'm facing the exact same issue. The only way I can get my amplify cli generated functions to get authorized is by setting the models publicly available, which is obviously not what I'm looking to do.

Lorenzohidalgo commented 2 years ago

The cloud formation template seems to display the correct access permissions:

        {
              "Effect": "Allow",
              "Action": [
                "appsync:GraphQL"
              ],
              "Resource": [
                {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:appsync:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":apis/",
                      {
                        "Ref": "<api_name>GraphQLAPIIdOutput"
                      },
                      "/types/Query/*"
                    ]
                  ]
                },
                {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:appsync:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":apis/",
                      {
                        "Ref": "<api_name>GraphQLAPIIdOutput"
                      },
                      "/types/Mutation/*"
                    ]
                  ]
                },
                {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:appsync:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":apis/",
                      {
                        "Ref": "<api_name>GraphQLAPIIdOutput"
                      },
                      "/types/Subscription/*"
                    ]
                  ]
                }
              ]
            },

The workaround explained in the issue aws-amplify/amplify-cli#6933 does not work for me, at least not while mocking the function.

The generateGraphQLPermissions flag is also enabled:

"appsync": {
  "generateGraphQLPermissions": true
}

anttironty commented 2 years ago

Adding that I get this following error message when running amplify mock and have any { allow: private, provider: iam } auth rules in my schema.

Error message:

Mock does not handle CloudFormation resource of type AWS::IAM::ManagedPolicy. Skipping processing resource AuthRolePolicy<policyId>

Additional info: I have the following auth rules set on my API through amplify cli. Cognito user pools (default), IAM, Api key

Not sure is this error message related to this issue, but to me it seems like it is.

edwardfoyle commented 2 years ago

Hi @Lorenzohidalgo after setting the generateGraphQLPermissions flag to true, rerun amplify update function and select the API permissions again. This should generate a policy that has access to the query/mutation/subscription endpoints of the API rather than the control plane APIs. Then run amplify push to update the function in the cloud

Lorenzohidalgo commented 2 years ago

Hi @edwardfoyle, the flag generateGraphQLPermissions was already set to true when the function was created.

I've tryed the following without any success:

Revoking and granting access to all resources
Revoking access, pushing, granting access, pushing again
Revoking all access, pushing, granting access only to api resource, pushing again

I would also like to add that the granted permissions to access other functions are correctly working.

Lorenzohidalgo commented 2 years ago

@edwardfoyle any update on this? It's somewhat of a big stopper for us.

Lorenzohidalgo commented 2 years ago

@edwardfoyle just to confirm, I've updated to Amplify CLI v8.0.0 and the issue still persists after pushing the function again.

I've also checked the function in the lambda GUI and it seems to recognize the permissions correctly:

Not sure where to continue checking/testing

UPDATE: The previous functions that were correctly working (and authorized with IAM) stopped working when I first pushed the new function.

Lorenzohidalgo commented 2 years ago

@edwardfoyle after more investigations and tests I've reached the following conclusions:

Pushing a function with amplify cli v7.6.26 removed the IAM access to AppSync. I suppose that if the IAM role is not specified as of the docs it now denies the access. I previously was able to query inside AppSyncs Console with my IAM.
Mocking a function (or testing it within Lambdas GUI) won't use the amplify defined IAM role. Not sure why, or if it's even intended. But mocking or testing the function ends up in an authentication error while running it once deployed ends up working properly.

Please confirm if this is the intended behavior.

fomson commented 2 years ago

Same here! Tried a lot of variations... Confirm what @Lorenzohidalgo describes.

Before updating to Transformer 2 everything worked fine - was able to query inside AppSyncs Console with my IAM.

Now, CANNOT query inside AppSyncs Console with my IAM [and calling from functions with IAM gives "message": "Not Authorized to access create on type " ].

This is a very blocking issue!!!

@edwardfoyle @sundersc any suggestions? :)

Lorenzohidalgo commented 2 years ago

@fomson to be able to query inside the AppSync Console with IAM you need to follow the steps mentioned in this section of the docs.

fomson commented 2 years ago

@fomson to be able to query inside the AppSync Console with IAM you need to follow the steps mentioned in this section of the docs.

Thank you! Will try now.

By any chance, has this ability to query inside the console solved the problem with Lambda authentication for you? [This is the main issue, really. All may Lamndas get denied accessing any resolvers...]

Lorenzohidalgo commented 2 years ago

@fomson if you create the functions as indicated in the docs (giving them access to your current API resources), they should have the necessary privileges to work.

The issue (at least in my case) is that mocking the function with the amplify cli or testing it via the Lambda GUI will result in authentication issues. Once deployed they will have the needed permissions. You can test it by invoking the deployed functions from another "test" function.

edwardfoyle commented 2 years ago

@Lorenzohidalgo local mocking will use the AWS credentials available on your local machine when executing the function. You'll need to grant this user/role access to the resources if you want to test those SDK calls locally.

However, executing the function in the console uses the same execution role as invoking the lambda in any other way so I would expect the permissions to be set correctly in that case

fomson commented 2 years ago

@Lorenzohidalgo thank you, still [was] not working... It feels something went very wrong after migrating to this Transformer v2...

@edwardfoyle seems as this https://github.com/aws-amplify/amplify-cli/issues/10130 and this aws-amplify/amplify-category-api#100 (@danrivett) and this https://github.com/aws-amplify/amplify-category-api/issues/544 and this https://github.com/aws/aws-appsync-community/issues/214 are very related to what I've been experiencing.

I tried adding to cli.json

...
"appsync": {
      "generategraphqlpermissions": true [and I tried false]
    },
...

Not helpful...

The issue is that my Lambda resolver gets "message": "Not Authorized to access create on type " when calling Amplify-generated resolvers from within itself. [So, I am able to set/allow permissions for my custom Lambda resolver but get issues when this custom resolver tries calling Amplify-generated resolver...]

Very weird behaviour, as it seems that my Lambda has permissions to Mutations [create is a mutation].

For now, I rolled back to Transformer v1, as this migration nonsense is very blocking [so far I spent seven days ducking with it].

sundersc commented 2 years ago

@fomson @Lorenzohidalgo - Have you tried this workaround? This makes the resolver to allow access to all the IAM roles on an account.

fomson commented 2 years ago

@sundersc

I haven't for two reasons:

I do not know what my "adminRoleNames" are. I don't know how to find out the names either. [Also, I am not very interested in calling from AppSync Console, if this "workaround" is meant to help with that]
In the VTL template things seem to make sense: The template [i.e. what Amplify generated for Amplify-generate resolver], I have
```
...
#if( $ctx.identity.userArn == $ctx.stash.unauthRole )
#set( $isAuthorized = true )
#end
...
```

When my customer revolver gets called, in its Event I have

...
identity: {
    accountId: 'XXX',
    cognitoIdentityAuthProvider: null,
    cognitoIdentityAuthType: 'unauthenticated',
    cognitoIdentityId: 'XXX',
    cognitoIdentityPoolId: 'XXX',
    sourceIp: [ 'XXX' ],
    userArn: 'arn:aws:sts::XXX:assumed-role/amplify-XXX-dev-XXX-unauthRole/CognitoIdentityCredentials',
    username: 'XXX:CognitoIdentityCredentials'
  },
...

So, supposedly, the condition in the VTL template for Amplify-generated resolver should be met, but it is not...

Do you know how I can log $ctx.stash.unauthRole to find out what its value is?

Am I meant to have value for cognitoIdentityAuthProvider?

Please have a look at this https://github.com/aws/aws-appsync-community/issues/214

This is the problem we are dealing with.

josefaidt commented 2 years ago

Hey @Lorenzohidalgo :wave: re-reading your original post, I noticed the example shown is copied from our docs site which includes an if/else statement that is a bit confusing for the use case:

if (apiKey) {
    req.headers["x-api-key"] = apiKey;
} else {
    const signer = new AWS.Signers.V4(req, "appsync", true);
    signer.addAuthorization(AWS.config.credentials, AWS.util.date.getDate());
}

Does your API also have API Key as an auth mechanism? If so, it is likely this truthy check is attempting to authorize requests using the API key rather than signing the request for IAM auth.

fomson commented 2 years ago

@josefaidt

The point is that with IAM, there is an issue. I use no apiKey.

fomson commented 2 years ago

https://github.com/aws-amplify/amplify-cli/issues/10141#issuecomment-1104211754 just tested, apiKey is not defined [at least for me], so the conditional works well, the signer gets assigned and addAuthorization() is called.

The search for answers continues... Transformer v2 [or migrating to it] causes problems with GraphQL...

Lorenzohidalgo commented 2 years ago

@josefaidt same as @fomson here. removed that part because it should (and is) always trying to access via IAM.

josefaidt commented 2 years ago

Hey @Lorenzohidalgo and @fomson I've spent some time on this with quite a few Lambda's using IAM auth to call AppSync and did notice one instance where I was getting unauthorized where the permissions were available, and after a few minutes it seemed to relieve itself -- potentially due to some permission propagation?

Using the following schema and Lambda code I am able to call the AppSync API using IAM auth:

enum Status {
  ACTIVE
  INACTIVE
}

enum PaymentInterval {
  MONTHLY
  QUARTERLY
  YEARLY
}

type Membership
  @model
  @auth(
    rules: [
      { allow: owner, ownerField: "id", operations: [create, read] }
      { allow: private, provider: iam, operations: [update, delete] }
    ]
  ) {
  id: ID!
  status: Status @default(value: "INACTIVE")
  paymentInterval: PaymentInterval
}

// amplify/backend/function/updatemembership/src/index.js
import crypto from '@aws-crypto/sha256-js'
import { defaultProvider } from '@aws-sdk/credential-provider-node'
import { SignatureV4 } from '@aws-sdk/signature-v4'
import { HttpRequest } from '@aws-sdk/protocol-http'
import { default as fetch, Request } from 'node-fetch'

const { Sha256 } = crypto
const AWS_REGION = process.env.AWS_REGION || 'us-east-1'

const MUTATION_UPDATE_MEMBERSHIP = /* GraphQL */ `
  mutation MUTATION_UPDATE_MEMBERSHIP($input: UpdateMembershipInput!) {
    updateMembership(input: $input) {
      id
      status
      paymentInterval
    }
  }
`

/**
 * @type {import('@types/aws-lambda').APIGatewayProxyHandler}
 */
export const handler = async (event) => {
  console.log(`EVENT: ${JSON.stringify(event)}`)

  const { id, status, interval } = event.queryStringParameters
  const variables = {
    input: {
      id,
      status,
      paymentInterval: interval,
    },
  }

  const endpoint = new URL(process.env.API_9966_GRAPHQLAPIENDPOINTOUTPUT)

  const signer = new SignatureV4({
    credentials: defaultProvider(),
    region: AWS_REGION,
    service: 'appsync',
    sha256: Sha256,
  })

  const requestToBeSigned = new HttpRequest({
    method: 'POST',
    headers: {
      host: endpoint.host,
    },
    hostname: endpoint.host,
    body: JSON.stringify({ query: MUTATION_UPDATE_MEMBERSHIP, variables }),
    path: endpoint.pathname,
  })

  const signed = await signer.sign(requestToBeSigned)
  const request = new Request(endpoint, signed)

  let statusCode = 200
  let body
  let response

  try {
    response = await fetch(request)
    body = await response.json()
    if (body.errors) statusCode = 400
  } catch (error) {
    console.log(error)
    statusCode = 500
    body = {
      errors: [
        {
          message: error.message,
        },
      ],
    }
  }

  return {
    statusCode,
    body: JSON.stringify(body),
  }
}

And our function-parameters.json file in the function's directory has the following:

{
  "permissions": {
    "api": {
      "9966": [
        "Mutation"
      ]
    }
  },
  "lambdaLayers": []
}

Finally, the function's CloudFormation template should also include the policy document for adding these permissions:

"AmplifyResourcesPolicy": {
  "DependsOn": [
    "LambdaExecutionRole"
  ],
  "Type": "AWS::IAM::Policy",
  "Properties": {
    "PolicyName": "amplify-lambda-execution-policy",
    "Roles": [
      {
        "Ref": "LambdaExecutionRole"
      }
    ],
    "PolicyDocument": {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Action": [
            "appsync:GraphQL"
          ],
          "Resource": [
            {
              "Fn::Join": [
                "",
                [
                  "arn:aws:appsync:",
                  {
                    "Ref": "AWS::Region"
                  },
                  ":",
                  {
                    "Ref": "AWS::AccountId"
                  },
                  ":apis/",
                  {
                    "Ref": "api9966GraphQLAPIIdOutput"
                  },
                  "/types/Mutation/*"
                ]
              ]
            }
          ]
        }
      ]
    }
  }
}

Can you confirm your functions have the CloudFormation block shown above and the necessary permissions? If so, can you try the sample snippet for calling AppSync with IAM?

Lorenzohidalgo commented 2 years ago

@josefaidt there is no file called function-parameters.json in the functions directory. There is a file called parameters.json but it's an empty json ({}) for all of my functions. Should this file be populated automatically?

fomson commented 2 years ago

@josefaidt @Lorenzohidalgo

Let me just offer another file name where permissions are mentioned: "FUNCTION_NAME-cloudformation-template.json"...

@josefaidt What your are suggesting seems to be not the problem, please see this https://github.com/aws-amplify/amplify-cli/issues/10141#issuecomment-1090177866

The issue posted by @Lorenzohidalgo is around "message":"Not Authorized to access getUsers on type Query"

AND his "cloud formation template seems to display the correct access permissions..." for type Query...

josefaidt commented 2 years ago

Hey @Lorenzohidalgo the function-parameters.json file should exist and is not ignored by git. If you execute amplify update function does this populate/create the file? However, if your CloudFormation template has the appropriate permissions (as noted earlier in this thread), then it should be okay. Looking at the sample GraphQL schema snippet provided in the original post, can you confirm whether the iam provider is specified on the auth rule?

This will authenticate using Cognito

@auth(rules: [{ allow: private }])

This will authenticate using IAM

@auth(rules: [{ allow: private, provider: iam }])

Lorenzohidalgo commented 2 years ago

Hi @josefaidt ,

Sorry for the confusion, the file function-parameters.json does exist (and was already completed). The CLI configures VSCode to hide that file from the tree.

I've not seen any difference in the functions behavior between:

@auth(rules: [{ allow: private }])

and

@auth(rules: [{ allow: private }, { allow: private, provider: iam }])

Just to clarify, as mentioned in one of my last comments the issue does only affect mocking and testing the functions.

The issue (at least in my case) is that mocking the function with the amplify cli or testing it via the Lambda GUI will result in authentication issues. Once deployed they will have the needed permissions. You can test it by invoking the deployed functions from another "test" function.

josefaidt commented 2 years ago

Hey @Lorenzohidalgo apologies for my confusion! Does the note here provide additional insight as to how to grant access to the API when mocking?

local mocking will use the AWS credentials available on your local machine when executing the function. You'll need to grant this user/role access to the resources if you want to test those SDK calls locally.

Essentially we'll want to mirror the permissions generated for our function to the IAM user used by Amplify CLI when mocking

johnf commented 2 years ago

I'm seeing something similar, not sure if it;s related though. I have a mutation which is getting errors when talking to the grapql API.

The lambda function is called by graphQL just fine. But when the lambda tries to talk to GrapqhQL it is getting an error.

What I've worked out is that in my local autogenerated VTL the function is listed in adminRoles, but it isn't in App Sync itself.

So the VTL isn't in sync.

I've tried a couple of things but can't work out how to resync the VTL files.

johnf commented 2 years ago

I ended up adding a dummy field to a table and pushing to get the VTL back in sync

josefaidt commented 2 years ago

What I've worked out is that in my local autogenerated VTL the function is listed in adminRoles, but it isn't in App Sync itself.

So the VTL isn't in sync.

I've tried a couple of things but can't work out how to resync the VTL files.

@johnf This is an interesting note. To clarify, what you were seeing locally upon running amplify api gql-compile was not matching the deployed copy of the VTL in the AppSync Console? If so, would you mind filing a separate bug report for this?

johnf commented 2 years ago

@josefaidt See https://github.com/aws-amplify/amplify-category-api/issues/679

josefaidt commented 2 years ago

Hey @johnf great callout and we were able to reproduce that issue per this comment https://github.com/aws-amplify/amplify-category-api/issues/679. @Lorenzohidalgo and @fomson can you check the linked issue here and see if the notes here provide additional insight? In short, granting access to an already-created API with no current, pending changes will not push the VTL resolver update allowing the function access. As a workaround we can make a small/empty change to our GraphQL schema to trigger an update to the API resource and push the changes.

Lorenzohidalgo commented 2 years ago

hey @josefaidt, I reviewed the comment and it seems to apply/be the source issue.

I just faced the issue:

created a new function (dynamoDB stream trigger)
On push only the function changes are detected
Faced authorization issues

Updating the function to remove and reapply the permissions did not work, but adding a blank line in the graphql schema and deploying again did the trick for me.

Thanks a lot for the tip @johnf !!

jeremyrajan commented 2 years ago

@fomson @Lorenzohidalgo - Have you tried this workaround? This makes the resolver to allow access to all the IAM roles on an account.

Tried the workaround but didn't work as well sadly. Is there a workaround that works? :)

josefaidt commented 2 years ago

Hey @Lorenzohidalgo glad to hear you're back up and running 🙌

@jeremyrajan what sort of issues are you running into?

josefaidt commented 2 years ago

Adding pending-close... label in favor of tracking aws-amplify/amplify-category-api#679

fomson commented 2 years ago

@josefaidt Why is this being closed if a clear solution hasn't been found, has it?

josefaidt commented 2 years ago

Hey @fomson apologies for not also adding a comment, but I've closed in favor of tracking the bug described in aws-amplify/amplify-category-api#679. As a workaround, we can add a small comment or space to our schema.graphql file to trigger an update for our API resource and push.

Weixuanf commented 1 year ago

Hi I experienced similar issue, for me it was because I didn't add IAM auth mode for my API (graphql) I only added API_KEY and COGNITO_USER_POOL, but not IAM for my API. To add IAM auth mode: amplify update API -> follow the prompt to update the authorization mode for your API :) then everything just works!

Some context on how did I get the problem:

amplify add function -> ... -> select Lambda trigger (triggered by dynamo db update event in my case)
amplify update function -> grant graphql API query and mutation permission for lambda function
inside lambda trigger I tried to do an mutation on a table but got denied with unauthorized error ( I followed code sample here: https://docs.amplify.aws/lib/graphqlapi/graphql-from-nodejs/q/platform/js/#iam-authorization ) After investigating and tried many methods in this thread, I had no luck, UNTIL, I tried to do amplify add function but select AppSync - GraphQL API request (with IAM) this time, I finally got the error "iam auth not enabled for appsync api. to update an api, use "amplify update API" Then I realized, ok it's because I didn't add IAM auth mode in my API!!

Once suggestion, in amplify update function, it should remind developer if the API haven't setup IAM authorization, it's a caveat that's easy to neglect

qwikag commented 1 year ago

Hi I experienced similar issue, for me it was because I didn't add IAM auth mode for my API (graphql) I only added API_KEY and COGNITO_USER_POOL, but not IAM for my API. To add IAM auth mode: amplify update API -> follow the prompt to update the authorization mode for your API :) then everything just works!

Hang on Hang on,.. But why!

Can someone please explain why "Cognito, API, and now IAM.

And if I do not have IAM users, instead I have IAM(sso) users how does that work.

Please guys tell how these are meant to be used and tell how to set them up.

Documentation on these topics is killing us.

MensurRasic commented 1 year ago

Same issue here, I am not able to perform AppSync calls from lambda functions and it was working perfectly fine before... the documentation is not providing any clear solution...

rafaelfaria commented 1 year ago

+1 for having this working before and now i am getting Not Authorized to access myFunction on type Mutation

MensurRasic commented 1 year ago

@rafaelfaria What worked for us, it was to insert a custom VTL script at the top the of the generated mutation using amplify override... push it remote... then remove it pipeline function we added... and push again... but it's not a viable solution to suggest if you never used amplify override.

aws-amplify / amplify-cli