How to access AWS AppSync via a Lambda function? (queries and mutations)

[janhesters]

Which Category is your question related to? API What AWS Services are you utilizing? Lambda, AppSync, Cognito Provide additional details e.g. code snippets I'm trying to access AWS AppSync via a Lambda function.

1. amplify init
2. amplify add auth
3. amplify add api -> GraphQL

type Todo @model @auth(rules: [{ allow: owner }]) {
  id: ID!
  name: String!
  description: String
}

4. amplify add funtion choose rest

/* Amplify Params - DO NOT EDIT
You can access the following resource attributes as environment variables from your Lambda function
var environment = process.env.ENV
var region = process.env.REGION
var apiSaymexampleGraphQLAPIIdOutput = process.env.API_SAYMEXAMPLE_GRAPHQLAPIIDOUTPUT

Amplify Params - DO NOT EDIT */

var express = require('express');
var bodyParser = require('body-parser');
var awsServerlessExpressMiddleware = require('aws-serverless-express/middleware');
var Amplify = require('aws-amplify');

// declare a new express app
var app = express();
app.use(bodyParser.json());
app.use(awsServerlessExpressMiddleware.eventContext());

// Enable CORS for all methods
app.use(function(req, res, next) {
  res.header('Access-Control-Allow-Origin', '*');
  res.header(
    'Access-Control-Allow-Headers',
    'Origin, X-Requested-With, Content-Type, Accept'
  );
  next();
});

var region = process.env.REGION;
var apiSaymexampleGraphQLAPIIdOutput =
  process.env.API_SAYMEXAMPLE_GRAPHQLAPIIDOUTPUT;

// get endpoint from AppSync console
var appSyncEndPoint =
  'https://xxxx.appsync-api.eu-central-1.amazonaws.com/graphql';

Amplify.default.configure({
  aws_appsync_graphqlEndpoint: appSyncEndPoint,
  aws_appsync_region: region,
  aws_appsync_authenticationType: 'AWS_IAM',
});

var listTodos = `query ListTodos(
  $filter: ModelTodoFilterInput
  $limit: Int
  $nextToken: String
) {
  listTodos(filter: $filter, limit: $limit, nextToken: $nextToken) {
    items {
      id
      name
      description
    }
    nextToken
  }
}
`;

app.post('/items', function(req, res) {
  // Add your code here
  console.log('Running /items');
  console.log('Request', req);
  Amplify.API.graphql(Amplify.graphqlOperation(listTodos))
    .then(function(data) {
      console.log('Success');
      res.json({
        success: 'post call succeed!',
        url: req.url,
        body: req.body,
        data,
      });
    })
    .catch(function(error) {
      console.log('Error');
      console.log(error);
      res.json({ error });
    });
});

app.post('/items/*', function(req, res) {
  // Add your code here
  res.json({ success: 'post call succeed!', url: req.url, body: req.body });
});

app.listen(3000, function() {
  console.log('App started');
});

// Export the app object. When executing the application local this does nothing. However,
// to port it to AWS Lambda we will create a wrapper around that will load the app from
// this file
module.exports = app;

5. amplify add api choose rest and the function you created.
6. Create an account and some todos for it.
7. Invoke the function.

It gets a error 401 response, so I added this additional policy:

"AppSyncInvokePolicy": {
    "DependsOn": [
        "LambdaExecutionRole"
    ],
    "Type": "AWS::IAM::Policy",
    "Properties": {
        "PolicyName": "appsync-invoke-policy",
        "Roles": [
            {
                "Ref": "LambdaExecutionRole"
            }
        ],
        "PolicyDocument": {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Effect": "Allow",
                    "Action": [
                        "appsync:GraphQL"
                    ],
                    "Resource": [
                        "myappsyncarn/*"
                    ]
                }
            ]
        }
    }
},

Which still didn't work and I got an 401 error. What can I do to access AppSync via Lambda?

[jkeys-ecg-nmsu]

Hard to tell with AppSync if you're actually unauthorized, sometimes it returns 401 when you've got a malformed input. The main thing I see is that you're not passing a 'variables' object containing your three parameters:

// Creating a post is restricted to IAM 
const createdTodo = await API.graphql({
  query: queries.createTodo,
  variables: {input: todoDetails},
  authMode: 'AWS_IAM'
});

https://aws-amplify.github.io/docs/js/api#aws-appsync-multi-auth

Is it sending an access key in your request?

[janhesters]

@jkeys-ecg-nmsu I'm not using any variables, because I'm querying 😊 I assume it has something todo, that the policy grants I_AM access, but the API endpoint is configured for Cognito User Pools.

[jkeys-ecg-nmsu]

You define three variables here:

var listTodos = `query ListTodos(
  $filter: ModelTodoFilterInput
  $limit: Int
  $nextToken: String
)

But you are half-correct, because these variables are not used ($nextToken) or auto-generated if not provided ($limit, $filter) if not provided by the generated resolver:

#set( $limit = $util.defaultIfNull($context.args.limit, 10) )
{
  "version": "2017-02-28",
  "operation": "Scan",
  "filter":   #if( $context.args.filter )
$util.transform.toDynamoDBFilterExpression($ctx.args.filter)
  #else
null
  #end,
  "limit": $limit,
  "nextToken":   #if( $context.args.nextToken )
"$context.args.nextToken"
  #else
null
  #end
}

If you needed to list more than ten elements you would need to pass in a $limit variable.

Not sure about the IAM problem.

[janhesters]

@jkeys-ecg-nmsu Yes that's correct.

[robert-moore]

If you want to still use the @auth decorators I set it up to run through Cognito authentication and then just create an "admin" user that I log in with on lambda programmatically: https://www.floom.app/blog/aws-appsync-with-lambda

[janhesters]

@robert-moore Great article, great website, great product. Props!

Only problem is that you have to hardcode you admin credentials into you Lambda function.

[houmark]

Sorry for hijacking here, @janhesters you could consider AWS Secrets Manager for the username/password and some of the other sensible credentials in the Lamda function.

Having said that, @robert-moore great blog post, and I actually tried making that work today. I found a few minor errors in the blog post, but I was able to correct them (I could pass them on in private if you'd like to tweak it), but after setting it all up and running it locally I get ResourceNotFoundException: User pool us-east-1_mypoolid does not exist.

Tripple checked of course (maybe actually 20-checked and tried all combos of user pool + client id). I checked the console, I updated some configuration there also to make sure ADMIN_NO_SRP_AUTH is allowed for the Cognito user pool. Thought maybe it was because I was running locally, so pushed to the cloud and then I get a completely different error:

{
    "errorType": "Error",
    "errorMessage": "fetch is not found globally and no fetcher passed, to fix pass a fetch for\n      your environment like https://www.npmjs.com/package/node-fetch.\n\n      For example:\n        import fetch from 'node-fetch';\n        import { createHttpLink } from 'apollo-link-http';\n\n        const link = createHttpLink({ uri: '/graphql', fetch: fetch });\n      ",
    "stack": [
        "Error: fetch is not found globally and no fetcher passed, to fix pass a fetch for",
        "      your environment like https://www.npmjs.com/package/node-fetch.",
        "",
        "      For example:",
        "        import fetch from 'node-fetch';",
        "        import { createHttpLink } from 'apollo-link-http';",
        "",
        "        const link = createHttpLink({ uri: '/graphql', fetch: fetch });",
        "      ",
        "    at warnIfNoFetch (/var/task/node_modules/apollo-link-http/lib/bundle.umd.js:76:15)",
        "    at Object.createHttpLink (/var/task/node_modules/apollo-link-http/lib/bundle.umd.js:93:5)",
        "    at Object.exports.createAppSyncLink (/var/task/node_modules/aws-appsync/lib/client.js:121:201)",
        "    at new AWSAppSyncClient (/var/task/node_modules/aws-appsync/lib/client.js:191:72)",
        "    at getAppSyncClient (/var/task/app.js:44:13)",
        "    at Object.<anonymous> (/var/task/app.js:69:16)",
        "    at Module._compile (internal/modules/cjs/loader.js:701:30)",
        "    at Object.Module._extensions..js (internal/modules/cjs/loader.js:712:10)",
        "    at Module.load (internal/modules/cjs/loader.js:600:32)",
        "    at tryModuleLoad (internal/modules/cjs/loader.js:539:12)"
    ]
}

I tried updating Node 8.x runtime to Node 10.x runtime but didn't change anything.

You got any idea what's wrong?

I'd really really like to make this work to be able to subscribe to created/updated records, and to keep consistency between my frontend and backend.

[janhesters]

@houmark You are right. If I were to implement @robert-moore 's solution, I would simply add environment variables using the Lambda console.

But as I mentioned above, I went with accessing DynamoDB directly, because why would you use AppSync if you can edit the DB directly.

PS: Would love to see the mistakes you spotted in Rob's post.

[houmark]

@janhesters I edited my post to explain why I want it to this to work through GraphQL. 1) Subscriptions. I'm not aware of a way to ensure that my web app will get the notification on a new item unless it's going through GraphQL and not straight to the DB. 2) To be able to have one create mutation that is used both by the frontend and the Lambda, so when something is changed, both ends will use the same GraphQL code.

(1) is 90% of the reason. If I could go straight to the DB and get subscriptions, then I'd probably drop GraphQL until there's an official more painless way of doing this inside Amplify (which may never happen).

[houmark]

@houmark You are right. If I were to implement @robert-moore 's solution, I would simply add environment variables using the Lambda console.

I'd probably use the Secrets Manager as you can make sure it sticks around even if you delete the Lamda function and re-create it for some reason and since it's likely that you may have 2-3-5 Lambda functions doing GrapHQL they can all share the creds.

PS: Would love to see the mistakes you spotted in Rob's post.

I can share those, they are minor, but I'd like to do it more directly than to spam this thread more, and right now I had to drop my work on refactoring this due to the above-mentioned errors and to move on with other stuff pending in my project. I had a Lambda already running DynamoDB directly and I was on the hunt to change that to do it through GraphQL.

[janhesters]

@houmark Thank you, both of your argements are solid. But, maybe we are miscommunicating. I'm don't want modify DynamoDB directly from my client side. I only want to do this in the Lambda function. If you are talking about using GraphQL's subscriptions on the client side, I'm curios, how are you using those subscriptions in your Lambda function? What use case do they solve in a function that only lives for a few seconds anyway?

That's a pretty great argument for using secrets manager. I think it is indeed superior to using env variables. At least until Amplify makes DynamoDB's resources available in amplify-meta.json.

[houmark]

@houmark Thank you, both of your argements are solid. But, maybe we are miscommunicating. I'm don't want modify DynamoDB directly from my client side. I only want to do this in the Lambda function. If you are talking about using GraphQL's subscriptions on the client side, I'm curios, how are you using those subscriptions in your Lambda function? What use case do they solve in a function that only lives for a few seconds anyway?

I am using strictly GraphQL in the frontend and I want to use GraphQL 100% (or as close to) in any Lambda function that ultimately needs to update the DB. The goal is to get Subscriptions about updates/creates in the frontend when the Lambda adds/update data. When I go directly to DynamoDB in the Lambda, I get no subscription event in the frontend so my UI will not be able to update accordingly. The Lambda does not care about Subscriptions, but in effect of "posting" data through GraphQL as another frontend client, subscriptions will trigger.

[janhesters]

@houmark Got it, thank you 🙏🏻

That would probably only work manually with streams and I agree using AppSync and Amplify's code is then 100% better.

[kaustavghosh06]

@janhesters Could you check your Lmbda Execution role and see if it has the correct permissions? Could you paste the policies attached to that lambda execution role out here?

[janhesters]

@kaustavghosh06 Here is the LambdaExecutionRole:

"LambdaExecutionRole": {
    "Type": "AWS::IAM::Role",
    "Properties": {
        "RoleName": {
            "Fn::If": [
                "ShouldNotCreateEnvResources",
                "saymexampleLambdaRole594ae87b",
                {
                    "Fn::Join": [
                        "",
                        [
                            "saymexampleLambdaRole594ae87b",
                            "-",
                            {
                                "Ref": "env"
                            }
                        ]
                    ]
                }
            ]
        },
        "AssumeRolePolicyDocument": {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Effect": "Allow",
                    "Principal": {
                        "Service": [
                            "lambda.amazonaws.com"
                        ]
                    },
                    "Action": [
                        "sts:AssumeRole"
                    ]
                }
            ]
        }
    }
},

There is also AmplifyResourcesPolicy which doesn't allow queries/mutations.

"AmplifyResourcesPolicy": {
    "DependsOn": [
        "LambdaExecutionRole"
    ],
    "Type": "AWS::IAM::Policy",
    "Properties": {
        "PolicyName": "amplify-lambda-execution-policy",
        "Roles": [
            {
                "Ref": "LambdaExecutionRole"
            }
        ],
        "PolicyDocument": {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Effect": "Allow",
                    "Action": [
                        "appsync:Create*",
                        "appsync:StartSchemaCreation",
                        "appsync:GraphQL",
                        "appsync:Get*",
                        "appsync:List*",
                        "appsync:Update*",
                        "appsync:Delete*"
                    ],
                    "Resource": [
                        {
                            "Fn::Join": [
                                "",
                                [
                                    "arn:aws:appsync:",
                                    {
                                        "Ref": "AWS::Region"
                                    },
                                    ":",
                                    {
                                        "Ref": "AWS::AccountId"
                                    },
                                    ":apis/",
                                    {
                                        "Ref": "apisaymexampleGraphQLAPIIdOutput"
                                    }
                                ]
                            ]
                        }
                    ]
                }
            ]
        }
    }
}

Both policies are auto generated when doing:

? Provide a friendly name for your resource to be used as a label for this category in the project:
test

? Provide the AWS Lambda function name:
test
? Choose the function template that you want to use:
Serverless express function (Integration with Amazon API Gateway)
? Do you want to access other resources created in this project from your Lambda function?
Yes
? Select the category
api
? Api has 2 resources in this project. Select the one you would like your Lambda to access
saymexample
? Select the operations you want to permit for saymexample (Press <space> to select, <a> to toggle all, <i> to invert selection)
create, read, update, delete

[robert-moore]

@houmark regarding the Cognito UserPool error: ResourceNotFoundException: User pool us-east-1_mypoolid does not exist.. I believe I ran into this before and the issue ended up being that my App Client that I was using for Lambda had a client secret. I created a Client without a secret and it worked fine. And regarding the fixes you made to the blog post, if you wouldn't mind sharing those that would be great. I could get the blog updated

I also switched to using AWS SSM to store the password parameters instead of using Lambda env variables:

// Get an Id Token (JWT) for the user
async function getCredentials() {
  const username = 'admin'
  const passwordRequest = await ssm.getParameter({
      Name: `/cognito/admin-password`,
      WithDecryption: true,
    }).promise()
  params.AuthParameters = {
    USERNAME: username,
    PASSWORD: passwordRequest.Parameter.Value,
  }
  return new Promise((resolve, reject) => {
    console.log("Initiating Auth")
    cognitoSP.adminInitiateAuth(params, (authErr, authData) => {
      if (authErr) {
        reject(authErr)
      } else if (authData === null) {
        reject("auth data is null")
      } else {
        console.log("Auth Successful")
        resolve(authData)
      }
    })
  })
}

[kaustavghosh06]

@janhesters Which version of the CLI are you using? We had a fix to fix the AppSync IAM policies out here - https://github.com/aws-amplify/amplify-cli/pull/1634 If you install the latest version and run amplify update function, you should be able to perform queries, mutations on the graphql API generated by the CLI.

[houmark]

@janhesters Which version of the CLI are you using? We had a fix to fix the AppSync IAM policies out here - #1634 If you install the latest version and run amplify update function, you should be able to perform queries, mutations on the graphql API generated by the CLI.

I'm getting a bit confused here. I did see that you can set access to the GraphQL API from a Lambda function, but what does that do exactly? In this thread using a Coginito user is explained and while I did not make that work yet, I'm curious to understand if there's another built in way, so that the Lambda function can write without a user at all due to these policies being added? It seems there's no documentation on this or I have not found it yet at least.

[houmark]

@houmark regarding the Cognito UserPool error: ResourceNotFoundException: User pool us-east-1_mypoolid does not exist.. I believe I ran into this before and the issue ended up being that my App Client that I was using for Lambda had a client secret. I created a Client without a secret and it worked fine.

Thanks, I will try again, but I did try both the existing Clients I had in Coginito, one with a secret and one without. Same error on both.

And regarding the fixes you made to the blog post, if you wouldn't mind sharing those that would be great. I could get the blog updated

It's very minor stuff, I will share ASAP.

I also switched to using AWS SSM to store the password parameters instead of using Lambda env variables:

// Get an Id Token (JWT) for the user
async function getCredentials() {
  const username = 'admin'
  const passwordRequest = await ssm.getParameter({
      Name: `/cognito/admin-password`,
      WithDecryption: true,
    }).promise()
  params.AuthParameters = {
    USERNAME: username,
    PASSWORD: passwordRequest.Parameter.Value,
  }
  return new Promise((resolve, reject) => {
    console.log("Initiating Auth")
    cognitoSP.adminInitiateAuth(params, (authErr, authData) => {
      if (authErr) {
        reject(authErr)
      } else if (authData === null) {
        reject("auth data is null")
      } else {
        console.log("Auth Successful")
        resolve(authData)
      }
    })
  })
}

Great stuff, will try to add that in my end also.

[houmark]

Here's an update after a long day of frustrating trial and error.

As I couldn't even make my Lambda get the user pool, I decided to try with the SSM to see if I at least could pull a credential. Even after adding policies to my Lambda for pulling SSM, I still didn't work.

After many hours of no progress, I decided to make a simple Lambda directly in the AWS UI and take small steps. Surprisingly, every step worked nicely. I think (I lost track of all my steps) I had to add the policy for pulling SSM parameters.

I then went back to the console and Amplify and made a Hello World function. I gave it access to ALL API's and functions in my entire Amplify project (not that many resources). I then replicated the code and added it bit by bit, again in the UI. The UI allowed me to do so as I had a small codebase. Eventually I had to redo those changes in my editor and push that, because I needed packages installed. Every step, SSM, Cognito and eventually reading and writing to my GraphQL endpoint worked and my mutations ended up as DynamoDB records. Awesome!

So many steps for so little. It would be great if this would be simplified and Amplify would handle this fully automatically without having to do all these steps. Until that (maybe) happens, I will refactor my code to be a small package locally that I can require in all my functions that needs to do GraphQL operations. Eventually it would be cool to add this to npm with the required manual instructions for how to get it all linked up.

What happened in my old function? I have no clue, but I am guessing some policies got corrupted and that prevented Amplify from giving the right access for that Lambda function. All this silently showing me that all was good.

Thanks @robert-moore for your initial tutorial and help on this. Let's sync up somewhere on the minor nitpicks on the article, so this can be easier and helpful for others.

[damianrr]

Guys, quick (and probably dumb) question: I can't even get the var Amplify = require('aws-amplify'); to work. When I put that in a lambda function (or locally in nodejs) I get this error:

START RequestId: 97250071-ba05-45b6-80c1-bc2f8040da0c Version: $LATEST
module initialization error: ReferenceError
    at Object.<anonymous> (/var/task/node_modules/youtube-iframe/index.js:50:3)
    at Module._compile (module.js:652:30)
    at Object.Module._extensions..js (module.js:663:10)
    at Module.load (module.js:565:32)
    at tryModuleLoad (module.js:505:12)
    at Function.Module._load (module.js:497:3)
    at Module.require (module.js:596:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/var/task/node_modules/@aws-amplify/analytics/lib/Providers/AmazonPersonalizeHelper/MediaAutoTrack.js:15:27)
    at Module._compile (module.js:652:30)
END RequestId: 97250071-ba05-45b6-80c1-bc2f8040da0c
REPORT RequestId: 97250071-ba05-45b6-80c1-bc2f8040da0c  Duration: 108.33 ms Billed Duration: 200 ms     Memory Size: 128 MB Max Memory Used: 94 MB  
module initialization error
ReferenceError

Any ideas?

[jkeys-ecg-nmsu]

Amplify is a client side library. What are you trying to accomplish with it from a Lambda that cannot be accomplished with either an HTTPS client like Axios or a more appropriate framework? Just trying to understand the use case.

[damianrr]

Basically I need to insert some rows in a DynamoDB table but I need to do it through graphql to trigger the subscriptions on the frontend. How could I do that? My main doubt with issuing a simple HTTP post request with the query is how to provide the authentication (where can I get it from?). Do you have an idea (snippet?!) on how I could do that?

Thanks

[undefobj]

Hello all - there's a lot going on in this thread so it's a bit hard for me to follow everything, but my understanding of the original ask "I'm trying to access AWS AppSync via a Lambda function" is that:

1. You have an AppSync endpoint
2. You have a Lambda function
3. You wish to have this Lambda function send a GraphQL mutation or query to this AppSync endpoint
4. This should work if the AppSync endpoint is configured either for IAM or API Key

Is my understanding correct? If not please correct me. If it is correct I believe one of your original issues is that you're trying to use the AmplifyJS library inside a NodeJS function. This is not supported at this time, and could be why you're seeing issues. However I created a sample function using Node's HTTP modules which you can see here: https://gist.github.com/undefobj/abb8a42a5c59606fb4126f47a383c48a

I want to stress this is a sample to get your feedback and see if it helps. Note that to use IAM on an AppSync endpoint you need to have appropriate Lambda execution role policy as defined HERE however the AppSync CLI should do this for you if you follow the add function flow after already going through add api.

The function has a module (query.js) where I'm exporting the GraphQL statement to run. I need to clean up some of the variable names and such, but if this works for your needs then we could either just offer this as a sample or add it to the CLI flow as a template. If we were to do that some questions:

1. Would adding all possible queries/mutations generated in a separate module be helpful?
2. If they were in a separate module how would you "select" which one to run? In code or at runtime via arguments?
3. Should any mutation/query arguments be an input to the Lambda that is read from the event object?
4. Anything else?

[damianrr]

Hey @undefobj, Thanks! That's pretty much what I was looking for! However, I'm having one problem with authentication: I'm getting this:

Response:
{
  "statusCode": 200,
  "body": {
    "data": {
      "createForecast": null
    },
    "errors": [
      {
        "path": [
          "createForecast"
        ],
        "data": null,
        "errorType": "Unauthorized",
        "errorInfo": null,
        "locations": [
          {
            "line": 2,
            "column": 5,
            "sourceName": null
          }
        ],
        "message": "Not Authorized to access createForecast on type Mutation"
      }
    ]
  }
}

Request ID:
"6bc5eead-09b4-4b00-b135-eb3dcf981b66"

Function Logs:
START RequestId: 6bc5eead-09b4-4b00-b135-eb3dcf981b66 Version: $LATEST
2019-06-21T15:04:46.024Z    6bc5eead-09b4-4b00-b135-eb3dcf981b66    { data: { createForecast: null },
  errors: 
   [ { path: [Array],
       data: null,
       errorType: 'Unauthorized',
       errorInfo: null,
       locations: [Array],
       message: 'Not Authorized to access createForecast on type Mutation' } ] }
END RequestId: 6bc5eead-09b4-4b00-b135-eb3dcf981b66
REPORT RequestId: 6bc5eead-09b4-4b00-b135-eb3dcf981b66  Duration: 2095.59 ms    Billed Duration: 2100 ms    Memory Size: 128 MB Max Memory Used: 84 MB  

I added by hand all AppSync permissions to the function role (hell, I even add AdministratorAccess) and also added IAM as additional authorization provider in AppSync (main one is Cognito) and still getting the same ... any ideas of what I might doing wrong?

Edit: Basically, the problem is that when I create the GraphQL API I selected Cognito User Pool as authentication method. Now if I switch appsync main authentication method to IAM the snippet works, but if I left Cognito User Pool as main authentication method and add IAM as additional ... it doesnt. But I can't leave it with IAM as main auth because then my amplify frontend doesnt work ... what can I do here?

Regarding your question, I'd go with 3 IMHO.

[undefobj]

@damianrr The code sample I made only works for IAM or API Key, you wouldn't want to use this with User Pools auth which it looks like per your edit you determined. What you need to do is add an additional auth provider in your AppSync API so that you have User Pools for your client apps and IAM for your Lambda function. More information: https://medium.com/@ednergizer/multiple-authorization-methods-in-a-single-graphql-api-with-aws-appsync-security-at-the-data-7feeaa968486 https://docs.aws.amazon.com/appsync/latest/devguide/security.html#using-additional-authorization-modes

[damianrr]

@undefobj, problem is that when I add IAM as additional (not main) auth provider in AppSync API then I also get "not authorized", and if I set it up as main and Cognito User Pools as additional then the UI client is the one that doesn't work :/

[undefobj]

@damianrr can you share your schema with the annotations?

[houmark]

@damianrr The code sample I made only works for IAM or API Key, you wouldn't want to use this with User Pools auth which it looks like per your edit you determined.

After some long serious struggles, I made my Lambda work with a Cognito user that is part of a specific group. While not the most perfect solution, I prefer that over IAM. I've been testing it now for a few days and it seems stable. node-fetch is patching AWSAppSyncClient to work on the server.

@undefobj I'm curious to understand more about why "you wouldn't want to use this with User Pools". Are there any serious downsides I did not discover. The node-fetch "patch" seems to be stable and used by many all over the interwebs.

[damianrr]

@undefobj here is my schema annotations for the models in question:

type Forecast                                       
  @model                    
  {                                                           
    id: ID! # REMEMBER: target datetime + created datetime
    items: [ForecastItem]  
    createdAt: String
    updatedAt: String                                                  
  }                                                                  

type ForecastItem {                       
  date: AWSDateTime!                                                                         
  formatted_date: String!                         
  enpowered: Float!                                                            
  ieso: Float!            
}

I tried adding @aws_iam @aws_cognito_user_pools as per the articles you mentioned above but then I get:

Schema Errors:
Unknown directive "aws_iam".
GraphQL request (201:19)
200: 
201: type ForecastItem @aws_iam {
                       ^
202:   date: AWSDateTime!

[damianrr]

I finally bailed on this approach, instead I took another route, which basically is authenticating in cognito, getting the IdToken and using it to make the graphql request, not the most elegant thing in the world, but hey, it works. I put a gist together in case it could be helpful to someone else: https://gist.github.com/damianrr/6cd78290c56b960bb2f4d543423b83a2 @houmark, I wonder if this is similar to the path you took.

Thank you @undefobj for all your support here.

[houmark]

@houmark, I wonder if this is similar to the path you took.

Yes very similar. I also use SSM for parameters, but I use getParameters to only do one call to SSM instead of 3, so I'd like to think it runs a bit faster vs. the await approach.

Interestingly you are using graphql-request which I have not considered. I am using AWSAppSyncClient from aws-appsync package.

I find this decently elegant and the only downside to the Cognito approach is if someone deletes the admin user by accident, but then you could also argue that they would potentially delete the IAM policy and so on...

This really should be part of the core of Amplify. I am not aware of one single API out there that can send webhooks to a GraphQL endpoint, and even if they could, I'm not sure how authentication would work for that. In our platform, some data is created and updated by third-party platforms that send webhooks and it's not really cool to not have live/socket updates for those situations.

[undefobj]

I tried adding @aws_iam @aws_cognito_user_pools as per the articles you mentioned above but then I get:

@damianrr Those directives must be added in the AppSync console. Amplify CLI doesn't support the multi-auth directives yet. That should be available in the next couple weeks as the PR is in process: https://github.com/aws-amplify/amplify-cli/pull/1524

[undefobj]

@undefobj I'm curious to understand more about why "you wouldn't want to use this with User Pools". Are there any serious downsides I did not discover. The node-fetch "patch" seems to be stable and used by many all over the interwebs.

@houmark You would need to authenticate via User Pools from the Lambda before you could make an API call. This isn't a good practice, it's better to use IAM from your backend Lambda and User Pools from your clients (or API key if they want public access).

[ajhool]

@undefobj

I'm having trouble following this thread, are these following things an accurate characterization of what you're saying?:

1. If you use amplify add api to configure a GraphQL AppSync API with "aws_appsync_authenticationType": "AMAZON_COGNITO_USER_POOLS", then the only way to make Amplify's API calls from an external/non-amplify lambda function is to first use Amplify's Auth module to login to a user? (eg. API.graphql(graphqlOperation(updateComment, { id:"some-comment-id", status: 'APPROVED-BY-ADMIN-LAMBDA' }));)

2. The only way to use the AppSync endpoint generated by Amplify is to access it via @amplify/api.graphql(graphqlOperation...). Can we simply use another GraphQL client in the lambda function to execute graphql queries/mutations?

3. My goal is to use cognito userpools to allow webapp users to interact with their Amplify resources, however, I would like to use lambda functions to perform Admin/Service activities on the objects (eg. change status). My strategy was to use Cognito in the frontend, but then simply grant IAM permissions to the execution role of the lambda functions, which would allow the lambda function to perform API.graphql(graphqlOperation(createComment, { input })); without logging in, first. Is that possible? Is it possible with a different graphql client?

Currently, I'm using SSM to store the username and password of a Cognito user in an group called "AdminService", however, AWS cognito rate-limits SignIns. Thus, if I want to execute a service lambda function that is watching a dynamoDB stream on a "Comment" table, and that service lambda function has to login every time a comment changes, then that will hit a hard Cognito limit really quickly. IAM is clearly the better way to do this.

@robert-moore are you sure that your solution is scalable? I'm currently using that approach (with SSM for secrets), but Cognito throttling would be brutal if the app started running at scale, I think.

[damianrr]

Guys, just to leave a bit of (successful) feedback: I managed to get this working with @undefobj's gist and by adding @aws_iam and @aws_cognito_user_pools in AppSync schema as in the articles he mentions above.

[ajhool]

Thanks @damianrr . Did you need to edit your AppSync schema through the console or can you do that in amplify/backend/api/schema.graphql? If you needed to edit it in the backend, do you see any errors when you run Amplify push? Does Amplify push overwrite your changes to the schema?

[damianrr]

@ajhool Yes, I edited the schema through the console, unfortunately as @undefobj mentioned above, Amplify CLI doesn't support multi auth directives yet (there's a PR about that here: https://github.com/aws-amplify/amplify-cli/pull/1524). amplify push will override these changes if you modify the schema, otherwise (changes to functions et al) it won't mess with it.

[undefobj]

Glad you got it working. To recap: Right now the CLI doesn't support those multi-auth directives but it's coming soon. In the meantime use the CLI with User Pools as the Authorization mode (which is what clients will use) then go into the AppSync console by running amplify console after the deployment completes, add in an additional Authorization Provider of IAM, and then annotate the schema in the console. Then you can use a Lambda to call that AppSync endpoint just make sure that Lambda has the appropriate execution role policy per my links above.

It's a couple extra hops right now but will be more streamlined once we merge these directives into the CLI.

[janhesters]

@undefobj Sorry I was on vacation over the last few days without internet.

1. You have an AppSync endpoint
2. You have a Lambda function
3. You wish to have this Lambda function send a GraphQL mutation or query to this AppSync endpoint
4. This should work if the AppSync endpoint is configured either for IAM or API Key

1-3 is correct. I don't know about 4. I want to make AppSync queries and mutations to the GraphQL API added by amplify add api and configured to auth using Cognito User Pools. And if you say 1, amplify add api, 2. choose REST, 3. create a new function and choose GraphQL API when it asks you which resources you wan't to access - it doesn't let you make requests from Lambda to your AppSync API.

Glad you got it working. To recap: Right now the CLI doesn't support those multi-auth directives but it's coming soon. In the meantime use the CLI with User Pools as the Authorization mode (which is what clients will use) then go into the AppSync console by running amplify console after the deployment completes, add in an additional Authorization Provider of IAM, and then annotate the schema in the console. Then you can use a Lambda to call that AppSync endpoint just make sure that Lambda has the appropriate execution role policy per my links above.

Looking forward to this PR. I will now try and follow those instructions above.

[undefobj]

@janhesters Why do you want to use a User Pool user in a Lambda function? This isn't good practice and is the whole reason we released the multi-auth feature in the first place. To do this you need to hardcode a fake username/password somewhere for that Lambda to use. You're better off using IAM for your Lambda.

[janhesters]

@undefobj I don't want to do that.

I want to access the AppSync GraphQL API, which handles the DynamoDB generated by amplify add api from a Lambda function. And I want to auth users in AppSync using Cognito User Pools (use @auth directive). I don't care which auth scheme the Lambda function uses to communicate with that API.

I want to use the same AppSync API from Lambda and the client side with Amplify to trigger subscriptions.

[janhesters]

@kaustavghosh06

If you install the latest version and run amplify update function, you should be able to perform queries, mutations on the graphql API generated by the CLI.

Using aws-sdk?

[jkeys-ecg-nmsu]

@undefobj can you please provide a code sample showing how to use IAM authorization with any generic http library like axios or fetch to an AppSync endpoint, i.e. how to sign your request manually if you don't want to drag Amplify into your backend?

A snippet from a schema that shows a model protected by both IAM and user pools would be greatly appreciated.

I'm not sure I follow the logic that a backend admin account with hardcoded credentials is inherently worse. Isn't that what Secrets Manager and KMS were designed to solve? Also encrypted Lambda environment variables. If you make your credentials as hard to break as the latest crypto algorithms then they seem equivalently secure. And you don't have to deal with multi-auth at that point simplifying your schema and knowledgebase.

[undefobj]

@jkeys-ecg-nmsu I provided a gist with this code above using standard Node HTTP calls: https://github.com/aws-amplify/amplify-cli/issues/1678#issuecomment-504287803

The docs links I provided as well as the blog post show multiple directives on a single API, just use @aws_iam where needed: https://github.com/aws-amplify/amplify-cli/issues/1678#issuecomment-504489578

I don't think this thread is the right place to debate pros/cons of hardcoding keys in a Lambda function. That being said best practice for a Lambda function is to use IAM policy on that execution role to interact with a destination service. This is provided in the links above. If you wish to hard code User Pool users into your Lambda function thats something that's always been possible and you can do that but I wouldn't recommend it.

[janhesters]

@undefobj

1. Would adding all possible queries/mutations generated in a separate module be helpful?

Yes. One has to write them anyway and Amplify is partly about generating boilerplate. You could also just add the mutations and queries that the given function has access to (e.g. just get and list if the function has readonly access).

2. If they were in a separate module how would you "select" which one to run? In code or at runtime via arguments?

I think keeping it flexible and having it in code would be the best.

3. Should any mutation/query arguments be an input to the Lambda that is read from the event object?

Yes. This would make running it locally easier.

4. Anything else?

Maybe automatically add the aws-sdk and a code example to the function. E.g. if you choose to access DynamoDB via the CRUD template, then it would be helpful to have similar examples for accesing AppSync from Lambda. Also have a way of list all items in the Lambda function, even though they would normally be filtered by user. There should also be a way to add the owner by getting the user in the Lambda function. The underlying question for the latter is: how do you get the owner (user) that calls the Lambda function?

[janhesters]

Update I tried this:

What you need to do is add an additional auth provider in your AppSync API so that you have User Pools for your client apps and IAM for your Lambda function.

But it won't work. I also checked the execution policy and the relevant permissions are there.

"AmplifyResourcesPolicy": {
    "DependsOn": [
        "LambdaExecutionRole"
    ],
    "Type": "AWS::IAM::Policy",
    "Properties": {
        "PolicyName": "amplify-lambda-execution-policy",
        "Roles": [
            {
                "Ref": "LambdaExecutionRole"
            }
        ],
        "PolicyDocument": {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Effect": "Allow",
                    "Action": [
                        "appsync:Create*",
                        "appsync:StartSchemaCreation",
                        "appsync:GraphQL",
                        "appsync:Get*",
                        "appsync:List*",
                        "appsync:Update*",
                        "appsync:Delete*"
                    ],
                    "Resource": [
                        {
                            "Fn::Join": [
                                "",
                                [
                                    "arn:aws:appsync:",
                                    {
                                        "Ref": "AWS::Region"
                                    },
                                    ":",
                                    {
                                        "Ref": "AWS::AccountId"
                                    },
                                    ":apis/",
                                    {
                                        "Ref": "apitodostreakGraphQLAPIIdOutput"
                                    },
                                    "/*"
                                ]
                            ]
                        }
                    ]
                }
            ]
        }
    }
}

Any ideas what I'm doing wrong?

PS: It works using directives:

type ModelTodoConnection @aws_iam
@aws_cognito_user_pools {
    items: [Todo]
    nextToken: String
}

type Query @aws_iam
@aws_cognito_user_pools {
    getTodo(id: ID!): Todo
    listTodos(filter: ModelTodoFilterInput, limit: Int, nextToken: String): ModelTodoConnection
    getStreak(id: ID!): Streak
    listStreaks(filter: ModelStreakFilterInput, limit: Int, nextToken: String): ModelStreakConnection
}

Problem here is that schemas generated with the @auth directive filter by owner in the .vtl template. Therefore the listQueries are always empty for an I_AM authenticated Lambda function.

[janhesters]

Another Update:

So I got it working both with the solution from @undefobj 's gist as well as using the regular AppSync client. To do the latter you need to add the following to your package.json.

"dependencies": {
  "apollo-cache-inmemory": "^1.1.0",
  "apollo-client": "^2.0.3",
  "apollo-link": "^1.0.3",
  "apollo-link-http": "^1.2.0",
  "aws-appsync": "^1.8.1",
  "aws-sdk": "^2.482.0",
  "aws-serverless-express": "^3.3.5",
  "body-parser": "^1.17.1",
  "es6-promise": "^4.2.8",
  "express": "^4.15.2",
  "graphql": "^0.11.7",
  "graphql-tag": "^2.10.1",
  "isomorphic-fetch": "^2.2.1"
},

And here is the code for a REST (express) Lambda function:

/* Amplify Params - DO NOT EDIT
You can access the following resource attributes as environment variables from your Lambda function
const environment = process.env.ENV
const region = process.env.REGION
const apiTodostreakGraphQLAPIIdOutput = process.env.API_TODOSTREAK_GRAPHQLAPIIDOUTPUT
const apiTodostreakGraphQLAPIEndpointOutput = process.env.API_TODOSTREAK_GRAPHQLAPIENDPOINTOUTPUT

Amplify Params - DO NOT EDIT */

const express = require('express');
const bodyParser = require('body-parser');
const awsServerlessExpressMiddleware = require('aws-serverless-express/middleware');
const AWS = require('aws-sdk');
const graphqlQuery = require('./graphql.js').query;
const gql = require('graphql-tag');
const AWSAppSyncClient = require('aws-appsync').default;
require('es6-promise').polyfill();
require('isomorphic-fetch');

const app = express();
app.use(bodyParser.json());
app.use(awsServerlessExpressMiddleware.eventContext());

app.use(function(req, res, next) {
  res.header('Access-Control-Allow-Origin', '*');
  res.header(
    'Access-Control-Allow-Headers',
    'Origin, X-Requested-With, Content-Type, Accept'
  );
  next();
});

const url = process.env.API_TODOSTREAK_GRAPHQLAPIENDPOINTOUTPUT;
const region = process.env.REGION;
const query = gql(graphqlQuery);

AWS.config.update({
  region,
  credentials: new AWS.Credentials(
    process.env.AWS_ACCESS_KEY_ID,
    process.env.AWS_SECRET_ACCESS_KEY,
    process.env.AWS_SESSION_TOKEN
  ),
});
const credentials = AWS.config.credentials;

const appsyncClient = new AWSAppSyncClient(
  {
    url,
    region: region,
    auth: {
      type: 'AWS_IAM',
      credentials,
    },
    disableOffline: true,
  },
  {
    defaultOptions: {
      query: {
        fetchPolicy: 'network-only',
        errorPolicy: 'all',
      },
    },
  }
);

app.get('/items', async function(_, res) {
  try {
    const client = await appsyncClient.hydrated();
    const data = await client.query({ query });
    res.json({ success: 'get call succeed!', data });
  } catch (error) {
    console.log(error);
    res.json({ error: 'get call failed!', error });
  }
});

app.post('/items', function(req, res) {
  res.json({ success: 'post call succeed!', url: req.url, body: req.body });
});

app.listen(3000, function() {
  console.log('App started');
});

module.exports = app;

But my problems still stand:

1. The Lambda function does not retrieve any items in this list query because of the way the .vtl template filters for owner. I'll probably add a new query and give it the @aws_iam directive, so that regular users can't use it and write this queries resolver without the filtering by owner.
2. Only granting permissions using directives works. Adding another default auth method using the AppSync console still leads to permission denied.

[attilah]

@janhesters It is the same thing I just replied in #1678: https://github.com/aws-amplify/amplify-cli/issues/1933#issuecomment-517410055

The multi-auth PR will solve this thing with the new resolvers.

I'm leaving this issue open until the #1916 will be merged.

[attilah]

Closing, as #1916 is merged and released.