Closed qwikag closed 7 months ago
Hey @qwikag :wave: thanks for raising this, and thanks for bringing this to our attention! We'll get those samples updated now that Lambda supports Node 18, removing the node-fetch
dependency for newly-created Functions.
Please note there is also an active issue we are tracking where Function access is not applied to your GraphQL API. This happens when the API is already deployed and you grant your Function access to the API to query/mutate, and the corresponding resolvers are not updated to include your Function's information for authorization, thus giving you a 401. To workaround this you can add an empty space to the GraphQL Schema to trigger an update and push with amplify push
Could we please get it rewritten to accurately describe how to set up a lambda's access to GraphQL API or via node-fetch or direct dynamo or something?
As a quick callout, it is not recommended to modify records on DynamoDB directly and rather query/mutate through GraphQL. This will ensure any DataStore-connected clients are notified of subscribed updates, and ensure the tables DataStore uses are updated.
What Amplify Version?
Functions created with Amplify CLI >12.2.x are created using the Node 18 runtime, and therefore can be built without the use of node-fetch
, however every recent version of Amplify CLI is able to create Functions with sufficient, scoped permissions for GraphQL operations (barring the issue and workaround noted above)
What libraries are available (node-fetch only), can API.GraphQL be used?
Unfortunately the aws-amplify
library does not currently support Node environments, however we will evaluate the feasibility of using it during the update of this doc you've linked.
Can it be setup with IAM/API-key/Cognito, and how?, and why?
IAM is our recommendation, however this can also be used with API Key with little updates needed for the code as outlined in the linked doc. For Cognito you will need to configure a REST API to use Cognito to authorize requests, which you can then send along to the GraphQL API request.
AWS SDK v3 - WHAT?
AWS SDK v3 is now included in Lambda's Node 18 runtime, superseding AWS SDK v2 that was available in previous runtime versions.
Proper debugging???
There is currently an active issue for debugging Lambda Functions with amplify mock
and using VSCode's debugging tools.
Policies to give access - WHAT?
IAM Policies for access are generated for you automatically when stepping through the amplify add|update function
flow and selecting "Grant Resource Access Permissions". No further action should be required from you with this flow.
Hi @josefaidt,
Fantastic response, Thank you.
I just want to clarify on this point:
This happens when the API is already deployed and you grant your Function access to the API to query/mutate
But the docs do say to do it that way:
The function can only be added when the GraphQL API with IAM authorization exists.
So can you please advise the best order of setting up a new function.
or
As a side I think this is the problem with most of the guides/turorials, there is very little step by step process. very little "why do we do this" and very little scenario based options (e.g. IAM(SSO), and App Router vs Pages.
The menu structure of the docs is not easy to find stuff. I think there a 2. ways of doing things. 1 via console and 2 via CLI, maybe others like configuration etc. and the console version should show how to do it but also advise that using the CLI guides is best practice. That way people will see the right outcome. e.g. what a VTLs and how do I know when they are setup correct.
Anyway, I hope to oneday be proficient enough to be more helpful/hands on.
Hey @qwikag
I just want to clarify on this point:
This happens when the API is already deployed and you grant your Function access to the API to query/mutate But the docs do say to do it that way: The function can only be added when the GraphQL API with IAM authorization exists.
This is a great callout! And I see how the callout in the docs shown after the example causes confusion. In the docs this is a callout specifically for the "AppSync - GraphQL API request (with IAM)" Function template you are prompted for on amplify add function
. For this we can move it above the code snippet to be more contextual
So can you please advise the best order of setting up a new function.
The second option here is fine. For the active issue I linked the only thing you'll want to ensure is that your API is updated after adding/updating a Function to have access to that API. You can inspect the auth
VTL resolvers to verify whether the Function name has been added as an authorized source.
As a side I think this is the problem with most of the guides/turorials, there is very little step by step process. very little "why do we do this" and very little scenario based options (e.g. IAM(SSO), and App Router vs Pages.
This and the following comments are also great callouts, and highlight a gap in our suggested workflows when considering both Amplify Hosting and the CLI. Let's continue this conversation in a docs issue. I see you have one regarding Next.js app router vs pages, but would you mind filing a new issue there with these thoughts?
@josefaidt This workaround doesn't work. I just created a new function and added the permissions through the CLI, added a space in my graphQL file and made the deployment with push and I'm still getting this error. The Amplify team should prioritize this type of bugs which have been reported since at least 1 year and 4 months ago, this basically makes all the documentation around 'using GraphQL on lambda' useless as if you follow it up it just doesn't work.
I would also asume by seeing the number of devs with issues on this matter that this affects a great deal of users, so why not prioritize this bugs which affect basic functionality for any type of app instead of adding shiny new things like passkeys? And I actually have a long list of things that the team could prioritize and that the devs have been asking for years like:
These are all things that could improve the developer experience greatly, and things that probably affect or could improve +80% of all projects done in Amplify.
@jerocosio
This workaround doesn't work. I just created a new function and added the permissions through the CLI, added a space in my graphQL file and made the deployment with push and I'm still getting this error.
>amplify push --force API
have you tried that?
Adding permission: what exactly did you do?
> amplify update api
Did you give IAM access to GraphQL API?
@qwikag I've tried almost everything found on Github/Stackoverflow by now, this bug has been messing with me now for 2 weeks, I decided to just query the database directly as there's no issues with getting permissions directly from the db, but once more I tried doing it from scratch and again got the same result, here are my detailed step-by-step:
amplify function add
and this were my options:
? Select which capability you want to add: Lambda function (serverless function)
? Provide an AWS Lambda function name: getItem
? Choose the runtime that you want to use: NodeJS
? Choose the function template that you want to use: AppSync - GraphQL API request (with IAM)
✅ Available advanced settings:
? Do you want to configure advanced settings? Yes ? Do you want to access other resources in this project from your Lambda function? Yes ? Select the categories you want this function to have access to. api ? Api has 2 resources in this project. Select the one you would like your Lambda to access xxxx ? Select the operations you want to permit on xxxx Query
You can access the following resource attributes as environment variables from your Lambda function API_XXXX_GRAPHQLAPIENDPOINTOUTPUT API_XXXX_GRAPHQLAPIIDOUTPUT API_XXXX_GRAPHQLAPIKEYOUTPUT ENV REGION ? Do you want to invoke this function on a recurring schedule? No ? Do you want to enable Lambda layers for this function? No ? Do you want to configure environment variables for this function? No ? Do you want to configure secret values this function can access? No ✔ Choose the package manager that you want to use: · NPM
2. Edited the `index.js` file of the generated `getItem` function and just copy/pasted the generated `listItems` query from GraphQL in src/graphlq, this is the file:
```javascript
/* Amplify Params - DO NOT EDIT
API_XXXX_GRAPHQLAPIENDPOINTOUTPUT
API_XXXX_GRAPHQLAPIIDOUTPUT
API_XXXX_GRAPHQLAPIKEYOUTPUT
ENV
REGION
Amplify Params - DO NOT EDIT */
import crypto from '@aws-crypto/sha256-js';
import { defaultProvider } from '@aws-sdk/credential-provider-node';
import { SignatureV4 } from '@aws-sdk/signature-v4';
import { HttpRequest } from '@aws-sdk/protocol-http';
import { default as fetch, Request } from 'node-fetch';
const GRAPHQL_ENDPOINT = process.env.API_XXXX_GRAPHQLAPIENDPOINTOUTPUT;
const AWS_REGION = process.env.AWS_REGION || 'us-east-1';
const { Sha256 } = crypto;
export const query = /* GraphQL */ `
query ListItems($filter: ModelItemFilterInput, $limit: Int, $nextToken: String) {
listItems(filter: $filter, limit: $limit, nextToken: $nextToken) {
items {
id
owner
public
}
nextToken
}
}
`;
/**
* @type {import('@types/aws-lambda').APIGatewayProxyHandler}
*/
export const handler = async (event) => {
console.log(`EVENT: ${JSON.stringify(event)}`);
const endpoint = new URL(GRAPHQL_ENDPOINT);
const signer = new SignatureV4({
credentials: defaultProvider(),
region: AWS_REGION,
service: 'appsync',
sha256: Sha256
});
const requestToBeSigned = new HttpRequest({
method: 'POST',
headers: {
'Content-Type': 'application/json',
host: endpoint.host
},
hostname: endpoint.host,
body: JSON.stringify({ query }),
path: endpoint.pathname
});
const signed = await signer.sign(requestToBeSigned);
const request = new Request(endpoint, signed);
let statusCode = 200;
let body;
let response;
try {
response = await fetch(request);
body = await response.json();
if (body.errors) statusCode = 400;
} catch (error) {
statusCode = 500;
body = {
errors: [
{
message: error.message
}
]
};
}
return {
statusCode,
// Uncomment below to enable CORS requests
// headers: {
// "Access-Control-Allow-Origin": "*",
// "Access-Control-Allow-Headers": "*"
// },
body: JSON.stringify(body)
};
};
Added one enter/space to the schema.graphql
for the Item
type so it looks like this:
type Item
@model(subscriptions: null)
@auth(rules: [{ allow: private, provider: iam }, { allow: owner }]) {
id: ID!
owner: String @auth(rules: [{ allow: owner, operations: [read, delete] }])
public: String @default(value: "false")
}
amplify status
and it looks like so:
┌──────────┬────────────────────────────────────┬───────────┬───────────────────┐
│ Category │ Resource name │ Operation │ Provider plugin │
├──────────┼────────────────────────────────────┼───────────┼───────────────────┤
│ Function │ getItem │ Create │ awscloudformation │
├──────────┼────────────────────────────────────┼───────────┼───────────────────┤
│ Api │ apiname │ Update │ awscloudformation │
├──────────┼────────────────────────────────────┼───────────┼───────────────────┤
│ Api │ AdminQueries │ No Change │ awscloudformation │
├──────────┼────────────────────────────────────┼───────────┼───────────────────┤
│ Auth │ authXXXX │ No Change │ awscloudformation │
├──────────┼────────────────────────────────────┼───────────┼───────────────────┤
│ Auth │ userPoolGroups │ No Change │ awscloudformation │
├──────────┼────────────────────────────────────┼───────────┼───────────────────┤
│ Function │ AdminQueries935fd287 │ No Change │ awscloudformation │
├──────────┼────────────────────────────────────┼───────────┼───────────────────┤
│ Function │ authXXXXPostAuthentication │ No Change │ awscloudformation │
├──────────┼────────────────────────────────────┼───────────┼───────────────────┤
│ Function │ authXXXXPreSignup │ No Change │ awscloudformation │
├──────────┼────────────────────────────────────┼───────────┼───────────────────┤
│ Storage │ XXXX │ No Change │ awscloudformation │
└──────────┴────────────────────────────────────┴───────────┴───────────────────┘
amplify push
and accepted all the changes, no issues and deployment succeeds.{
"statusCode": 400,
"body": "{\"data\":{\"listItems\":null},\"errors\":[{\"path\":[\"listItems\"],\"data\":null,\"errorType\":\"Unauthorized\",\"errorInfo\":null,\"locations\":[{\"line\":3,\"column\":5,\"sourceName\":null}],\"message\":\"Not Authorized to access listItems on type Query\"}]}"
}
If I go into the details of the lambda I can even see that the role is created and it has the necessary permissions or it looks like it but doesn't work:
Any suggestion greatly appreciated.
@jerocosio I went down the same path as you; your experience is not your fault, the documentation needs to be fixed and the tools also need fixing and I believe AWS are on top of that. Hence me helping you out so they can keep working on the issues.
But... You kinda ignored my questions... I am happy to help, but if you don't answer the helpers questions then you are only hurting yourself.
I will reiterate:
> amplify update api
(see below to include IAM)
update schema (add the space if need be or create more/less schema.
>amplify push --force api
I also have used
>amplify api gql-compile
&
> amplify codegen
But I think they are included in the push. (very little documentation)
Here is my setup:
> amplify update api
? Select from one of the below mentioned services: GraphQL
General information
- Name: XXXXX
- API endpoint: https://XXXXX.appsync-api.ap-southeast-2.amazonaws.com/graphql
Authorization modes
- Default: Amazon Cognito User Pool
- IAM
Conflict detection (required for DataStore)
- Disabled
? Select a setting to edit (Use arrow keys)
Authorization modes
@jerocosio Seemingly we do not need to use the @aws-sdk anymore. changing that to standard GrpahQL api would be interesting. personally I am in no hurry to fux it.
Your code looks exactly like mine and mine is now working due to the a space & --force api
So it is doable.
Thank you so much @qwikag for looking into this I really appreciate it, I continued following the steps you shared by doing:
amplify update api
to double check my settings for the authorization:
? Select from one of the below mentioned services: GraphQL
General information
Authorization modes
Conflict detection (required for DataStore)
? Select a setting to edit Authorization modes ? Choose the default authorization type for the API Amazon Cognito User Pool Use a Cognito user pool configured as a part of this project. ? Configure additional auth types? Yes ? Choose the additional authorization types you want to configure for the API API key, IAM API key configuration ✔ Enter a description for the API key: · xxxApiKey ✔ After how many days from now the API key should expire (1-365): · 365
2. Added one more line on the graphql file
3. Ran `amplify api gql-compile`
4. Ran `amplify codegen`
5. Updated the function to add a new console.log to debug more
6. Ran `amplify push --force api`
7. Ran a test on the lambda aws console
But still got the same error:
``` javascript
{
"statusCode": 400,
"body": "{\"data\":{\"listItems\":null},\"errors\":[{\"path\":[\"listItems\"],\"data\":null,\"errorType\":\"Unauthorized\",\"errorInfo\":null,\"locations\":[{\"line\":3,\"column\":5,\"sourceName\":null}],\"message\":\"Not Authorized to access listItems on type Query\"}]}"
}
I also just found something odd on my resolvers, on all of them in the auth I have this:
#if( $util.authType() == "IAM Authorization" )
#set( $adminRoles = ["xxxxPostConfirmation-dev"] )
#foreach( $adminRole in $adminRoles )
#if( $ctx.identity.userArn.contains($adminRole) && $ctx.identity.userArn != $ctx.stash.authRole && $ctx.identity.userArn != $ctx.stash.unauthRole )
#return($util.toJson({}))
#end
#end
#if( ($ctx.identity.userArn == $ctx.stash.authRole) || ($ctx.identity.cognitoIdentityPoolId == "us-west-1:xxxxx" && $ctx.identity.cognitoIdentityAuthType == "authenticated") )
#set( $isAuthorized = true )
#end
#end
But I actually deleted that function 'xxxxPostConfirmation' a long time ago, but I don't see the actual function that I'm trying to give access to anywhere in the resolvers.
@jerocosio Sounds like you have done all you can do.
I am on amplify-cli version 12.5.0
Lets discuss alternative platforms shall we.
I have just raised 2 very serious bugs occurring in the schema: https://github.com/aws-amplify/amplify-studio/issues/1044 https://github.com/aws-amplify/amplify-studio/issues/1045
there is no way to get proper attention here, or quick support. I have paid for support and as much as the people I have been dealing with are quality I think AWS have dug themselves into a hole. each time I have been the resolver of the bug (by luck or skill), and they have just played the role of pushing me to investigate more things.
So again what is a better platform outside AWS?
@qwikag I really like Amplify ad have used it in a number of projects, but it's true that support is basically non-existent and you have to find 'hacks' around making it work the way you want definately supabase or firebase are great alternatives to it, but I haven't really used them for a project lately.
I believe the issue with Amplify is the way they prioritize stuff, focusing on creating flashy stuff like their Figma integration, the Amplify Studio, etc... which are cool to have, but not really useful for more than 5% of their users vs things such as simple improvements to their transforms, functions, etc... (as mentioned above) which no one really sees but would improve 99% of their users.
Hey @jerocosio :wave: the error in the issue you're experiencing seems related to a missing @auth
rule on the Items
model. If the Function does not have access to call the API you should see a generic 401 or 403 response, where the 400 you're seeing is admittedly a bit confusing since it is also an auth-related error but from within the API. Can you share a snippet of your schema?
For example I've created a sample project using the following steps:
amplify init -y
amplify add api
> GraphQL > add both API Key and IAM for authamplify push -y
(to simulate having an existing API in the project when we add our function)
type Todo
@model
@auth(rules: [{ allow: public }, { allow: private, provider: iam }]) {
id: ID!
name: String!
description: String
}
amplify add function
AppSync - GraphQL API request (with IAM)
template? Which setting do you want to update? Resource access permissions
? Select the categories you want this function to have access to. api
? Select the operations you want to permit on 13235 Query, Mutation
amplify push -y
A few notes:
amplify add|update function
you should see an "AmplifyResourcesPolicy" added to the Function's CloudFormation template
"AmplifyResourcesPolicy": {
"DependsOn": [
"LambdaExecutionRole"
],
"Type": "AWS::IAM::Policy",
"Properties": {
"PolicyName": "amplify-lambda-execution-policy",
"Roles": [
{
"Ref": "LambdaExecutionRole"
}
],
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"appsync:GraphQL"
],
"Resource": [
{
"Fn::Join": [
"",
[
"arn:aws:appsync:",
{
"Ref": "AWS::Region"
},
":",
{
"Ref": "AWS::AccountId"
},
":apis/",
{
"Ref": "api13235GraphQLAPIIdOutput"
},
"/types/Query/*"
]
]
},
{
"Fn::Join": [
"",
[
"arn:aws:appsync:",
{
"Ref": "AWS::Region"
},
":",
{
"Ref": "AWS::AccountId"
},
":apis/",
{
"Ref": "api13235GraphQLAPIIdOutput"
},
"/types/Mutation/*"
]
]
}
]
}
]
}
#set( $adminRoles = ["myfunction-dev"] )
## [Start] Authorization Steps. **
$util.qr($ctx.stash.put("hasAuth", true))
#set( $isAuthorized = false )
#set( $primaryFieldMap = {} )
#if( $util.authType() == "API Key Authorization" )
#set( $isAuthorized = true )
#end
#if( $util.authType() == "IAM Authorization" )
#set( $adminRoles = ["myfunction-dev"] )
#foreach( $adminRole in $adminRoles )
#if( $ctx.identity.userArn.contains($adminRole) && $ctx.identity.userArn != $ctx.stash.authRole && $ctx.identity.userArn != $ctx.stash.unauthRole )
#return($util.toJson({}))
#end
#end
#if( !$isAuthorized )
#if( $ctx.identity.userArn == $ctx.stash.authRole )
#set( $isAuthorized = true )
#end
#end
#end
#if( !$isAuthorized && $util.isNull($ctx.stash.authFilter) )
$util.unauthorized()
#end
$util.toJson({"version":"2018-05-29","payload":{}})
## [End] Authorization Steps. **
@qwikag thank you for helping out here! Would you be open to hopping on a quick call to gather additional feedback? If so would you mind sending me a message at amplify-cli@amazon.com?
@jerocosio apologies, just noticed the schema snippet a few comments above,
type Item
@model(subscriptions: null)
@auth(rules: [{ allow: private, provider: iam }, { allow: owner }]) {
id: ID!
owner: String @auth(rules: [{ allow: owner, operations: [read, delete] }])
public: String @default(value: "false")
}
which looks good. The auth resolver you noted does look a bit odd. Is this an overridden resolver by chance? If you run amplify api gql-compile
and inspect the *.auth.*
resolver generated in amplify/backend/api/<api-name>/build/resolvers/Query.listItems.auth.1.req.vtl
does it still show the deleted function?
Hi @josefaidt, sending you email now.
@josefaidt thank you for looking into this, I just ran amplify api gql-compile
once more on my project and went through my resolvers and yeah the deleted function is still there, it's like my resolvers are not re-generating at all, and no it's not an overwritten one they're the ones generated from amplify. Is there anyway to force the re-generation of the resolvers? I can see the deleted function also on the resolvers inside the build directory.
@josefaidt I looked a little deeper into the files and they haven't been updated since at least a couple of weeks ago, I tracked down those dates and it looks like they match to when I created a new env for my project, not sure if it's 100% correlated, but looks like so.
@jerocosio can you try deleting the build directory and pushing?
@josefaidt I finally got it working, for some reason there was a resolver directory on the root of the api directory which had all the resolvers that I had in the past, so on the new build they were being picked as if they were made to override the new resolvers that were created. Thanks for digging into this and for the help, I hope that the overal bug around having to make edits on the schema every time a user wants to add/remove the permissions gets solved soon.
Hey @jerocosio glad to hear it!! That sounds like a side effect of an older bug where some mock issue would leave the resolvers/
directory in-tact with the generated resolvers https://github.com/aws-amplify/amplify-category-api/issues/1211
Any auth change I make is not reflected until I delete the resolvers folder and re mock. Huge hinderance to dev/debugging workflow.
Closing in favor of tracking existing item
Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.
How did you install the Amplify CLI?
Who Knows, because the process is all over the shop.
If applicable, what version of Node.js are you using?
20.3.1
Amplify CLI Version
12.4.0
What operating system are you using?
WIN 10
Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.
No manual changes
Describe the bug
This document is not helpful in any way. And this is a BUG! https://docs.amplify.aws/guides/functions/graphql-from-lambda/q/platform/js/
Because presently the process and commands to deliver on it do not work. So in rewriting this document and testing the process, the bugs should get removed in the process.
Could we please get it rewritten to accurately describe how to set up a lambda's access to GraphQL API or via node-fetch or direct dynamo or something? So that we can actually get our apps running!!!
The document needs to provide accurate steps, and actually work.
Also update the page with a date so we know when it was last updated/reviewed.
Expected behavior
the documented process takes the user through setting up and provides a few various common situations like Cognito triggers etc.
Reproduction steps
follow the current page: https://docs.amplify.aws/guides/functions/graphql-from-lambda/q/platform/js/
Project Identifier
1aff16e72d6657e6251bd6a212fd28ac
Log output
No response
Additional information
Points of interest and questions: What Amplify Version? What libraries are available (node-fetch only), can API.GraphQL be used? Can it be setup with IAM/API-key/Cognito, and how?, and why? AWS SDK v3 - WHAT? Proper debugging??? Policies to give access - WHAT?
Before submitting, please confirm: