Comments: Test the APIs

[jayair]

Link to chapter - http://serverless-stack.com/chapters/test-the-apis.html

[Spartano]

@samxi the syntax of --invoke is wrong, and double check the .yml syntax with the one from the guide.

apig-test --username admin@example.com --password Passw0rd! --user-pool-id YOUR_COGNITO_USER_POOL_ID --app-client-id YOUR_COGNITO_APP_CLIENT_ID --cognito-region YOUR_COGNITO_REGION --identity-pool-id YOUR_IDENTITY_POOL_ID --invoke-url YOUR_API_GATEWAY_URL --api-gateway-region YOUR_API_GATEWAY_REGION --path-template notes --method POST --body "{\"content\":\"hello world\",\"attachment\":\"hello.jpg\"}"

[samxi]

@jayair @Spartano

Rolling through all of the files/settings again I found a couple discrepancies and was able to get things working properly. Reading through these comments and all of your suggestions has helped me better understand the whole system and improve my troubleshooting skills.

Thank you both for your quick responses and help!

[nvamshik]

@jayair I am getting following error: Authenticating with User Pool Getting temporary credentials Making API request { status: 403, statusText: 'Forbidden', data: { message: 'Forbidden' } } I've tried everything (double checking my .YML file, policy, policy simulating, etc) but nothing works for me. Can you please help me with this issue? Following is my github repo: https://github.com/nvamshik/notes-app-api

[Spartano]

@nvamshik what OS are you using?

[nvamshik]

@Spartano I'm using Windows 8.1

[Spartano]

Did you use the syntax for windows users from the guide? Can you paste it here please.

[nvamshik]

I used the windows syntax. apig-test --username admin@example.com --password Passw0rd! --user-pool-id us-east-1_TXj0SxE --app-client-id 3201ojaiiav0k0p5ais0op3m --cognito-region us-east-1 --identity-pool-id us-east-1:c3c3e453-c386-4e59-b689-525bebc07 --invoke-url https://knl827x1si.execute-api.us-east-1.amazonaws.com/prod --api-gateway-region us-east-1 --path-template notes --method POST --body "{\"content\":\"hello world\",\"attachment\":\"hello.jpg\"}"

[jayair]

@nvamshik I'm not sure if it's a typo but the --path-template that we use in the tutorial is /notes not notes.

[quantuminformation]

I also get IdentityPool not found

Authentication is fine:

➜  serverless-stack-demo-api git:(master) ✗ apig-test \
--username='admin@example.com' \ 
--password='Passw0rd!' \ 
--user-pool-id='eu-west-2_achS8baFu' \
--app-client-id='6o3iivoaqbslnqatt46jche42e' \
--cognito-region='eu-west-2' \ 
--identity-pool-id='us-west-2:55f8e2df-7e47-4120-bdc0-843f2974f63b' \
--invoke-url='https://y85xk63wnk.execute-api.eu-west-2.amazonaws.com/prod' \
--api-gateway-region='eu-west-2' \
--path-template='/notes' \
--method='POST' \
--body='{"content":"hello world","attachment":"hello.jpg"}'
Authenticating with User Pool
Getting temporary credentials
IdentityPool 'us-west-2:55f8e2df-7e47-4120-bdc0-843f2974f63b' not found.

On AWS my pool is:

us-west-2:55f8e2df-7e47-4120-bdc0-843f2974f63b

arn:aws:cognito-identity:us-west-2:180971085012:identitypool/us-west-2:55f8e2df-7e47-4120-bdc0-843f2974f63b

Do pools and Identities need to be in the same region?

[quantuminformation]

I moved the Id pool to eu-west-2 and new error: Invalid identity pool configuration. Check assigned IAM roles for this pool. I will investigate and report back

[quantuminformation]

this is my permissions for the role assigned to this Id pool:

[jayair]

@QuantumInformation Can you following the steps in this comment (https://github.com/AnomalyInnovations/serverless-stack-com/issues/112#issuecomment-317204545) to see what the policy is for your Identity Pool? And post it here.

[quantuminformation]

Sure The Policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "mobileanalytics:PutEvents",
                "cognito-sync:*",
                "cognito-identity:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": [
                "arn:aws:s3:::notes-testtut/${cognito-identity.amazonaws.com:sub}*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "execute-api:Invoke"
            ],
            "Resource": [
                "arn:aws:execute-api:eu-west-2:*:y85xk63wnk/*"
            ]
        }
    ]
}

I tested the policy against the s3 bucket, but it said access is denied:

The bucket exists: http://notes-testtut.s3.amazonaws.com/ I checked the policy for the api and it has access.

[jayair]

@QuantumInformation I might be a little confused here, so your React app is able to make requests to your API but you are not able to upload files to S3?

[clarkgrg]

@jayair - I think some of these 403 and 500 errors are related to having aws profiles different than the default profile. I've redone this backend for the 3rd time today. First 2 goes I was not using the default profile in aws. This last time I set the aws configuration as the default. This time I didn't have a problem.

I hope this helps someone. Great tutorial really learning incredible amounts.

[quantuminformation]

So the error I was having was due to the role not having the correct trusted entity or smth.

Now I am getting this after fixing the string condition to match the cognito id:

Authenticating with User Pool
Getting temporary credentials
Making API request
{ status: 500,
  statusText: 'Internal Server Error',
  data: { status: false } }
➜  serverless-stack-demo-api git:(master) 

[quantuminformation]

@clarkgrg does apig-test use your profile? I don't think it does, it just behaves as a real user.

[quantuminformation]

Detailed error: I think I can fix this:

message: 'User: arn:aws:sts::180971085012:assumed-role/notes-app-api-prod-eu-west-2-lambdaRole/notes-app-api-prod-create is not authorized to perform: dynamodb:PutItem on resource: arn:aws:dynamodb:us-east-1:180971085012:table/notes',

[quantuminformation]

Ok I fixed it and its working now!

[jayair]

@QuantumInformation Do you mind leaving a comment saying what the issue you were seeing was and what you fixed. I think it'll help other folks. Thanks.

[quantuminformation]

Will do, btw I'm live streaming this tut: https://go.twitch.tv/quantuminformation#

[jayair]

@QuantumInformation That's awesome. I'm going to tweet it out.

[quantuminformation]

to fix my error I just had to update the policy file with the new location of the dyn DB table as it was the wrong region.

[clarkgrg]

I don't believe apig-test uses your profile. I think only serverless deploy. If you use profiles serverless deploy has a --profile (or --profile-name) option, but I struggled to get that going.

[aidaamp]

@jayair @marshallbunch I was able to run the apig-test on Windows PowerShell. I used the following code and it worked: apig-test --username admin@example.com --password Passw0rd! --user-pool-id YOUR_COGNITO_USER_POOL_ID --app-client-id YOUR_COGNITO_APP_CLIENT_ID --cognito-region YOUR_COGNITO_REGION --identity-pool-id YOUR_IDENTITY_POOL_ID --invoke-url YOUR_API_GATEWAY_URL --api-gateway-region YOUR_API_GATEWAY_REGION --path-template /notes --method POST --body '{""content"":""hello world"",""attachment"":""hello.jpg""}'

[chaosdarky]

Can someone help me? I'm not sure where did I go wrong but when I run apig-test, I'm getting a 500 error failed status. Upon checking the console log, i'm getting this validation error:

[ValidationException: One or more parameter values were invalid: Type mismatch for key userId expected: S actual: NULL] message: 'One or more parameter values were invalid: Type mismatch for key userId expected: S actual: NULL', code: 'ValidationException', time: Sun Oct 22 2017 09:17:00 GMT+0000 (UTC), requestId: 'G4GBGG9CUEN1DSU7EH809DN6KJVV4KQNSO5AEMVJF66Q9ASUAAJG', statusCode: 400, retryable: false, retryDelay: 8.820808271411806 }

It seems like that the event context that is being passed in the function has null userId. Based on the Identity pool console, I can see there that there's new identity but I don't know why I am getting a null userId.

Hope someone can help me with this.

[jayair]

@chaosdarky Is it the event.requestContext.identity.cognitoIdentityId that is null for you? Can you try to console.log and check which part of the event object is null?

[akolybelnikov]

Would someone help to debug an error? I have tried debugging, going through the whole process time after time, searching for solutions with AWS forums, but I'm still stuck on this one. I am getting

{ status: 502,
  statusText: 'Bad Gateway',
  data: { message: 'Internal server error' } }

response. here is the error from the log:

AccessDeniedException: User: arn:aws:sts::297650182091:assumed-role/bakery-api-prod-eu-central-1-lambdaRole/bakery-api-prod-create is not authorized to perform: dynamodb:PutItem on resource: arn:aws:dynamodb:eu-central-1:297650182091:table/products

(I have called my table and the app differently than in the tutorial). What am I missing and where is this coming from? The API are working locally.

[chaosdarky]

@jayair yes, the event.requestContext.identity.cognitoIdentityId is null. I tried to console log the event and the cognitoId there is null.

[akolybelnikov]

I resolved the AccessDeniedException by attaching a new inline policy for DynamoDB to my lambda function. The response status is 200 now. However, the issue with the status: 502 persists. I'm pretty sure that I have the same callback as in the tutorial and JSON.stringify, but I am having the Execution failed due to configuration error: Malformed Lambda proxy response in the CloudWatch and cannot find what's the reason for it. Anyone?

[jayair]

@akolybelnikov That usually happens when the response object is not formatted properly. You can debug this by logging what this line returns - https://github.com/AnomalyInnovations/serverless-stack-demo-api/blob/master/create.js#L20.

[MatthewTighe]

Re: Unexpected input for JSON body argument on Windows. Here's what worked for me:

'{\"content\":\"hello-world\",\"attachment\":\"hello.jpg\"}'

Doesn't seem to like the space in 'hello world'. I'm sure there's a way to escape it though.

[akolybelnikov]

@jayair While testing locally, it returns the 200 status like so:

{
    "statusCode": 200,
    "headers": "{\"Access-Control-Allow-Origin\":\"*\",\"Access-Control-Allow-Credentials\":true}",
    "body": "{\"userId\":\"USER-SUB-1234\",\"productId\":\"011fce60-bb57-11e7-9592-2599999d8bb3\",\"category\":\"buns\",\"productName\":\"zimt-bun\",\"content\":\"Christmas flavoured airy bun\",\"attachment\":\"zimtbun.jpg\",\"price\":\"133.10 RUB\",\"createdAt\":1509136801350}"
}

It's when I'm testing it remotely that I get this back:

{ status: 502,
  statusText: 'Bad Gateway',
  data: { message: 'Internal server error' } }

The thing is that the data is being posted in the DynamoDB: I can see it. I have tried a lot of things, but I cannot get to the root of the problem. What's the way to see what exactly causes this behaviour? I don't get to see much more info at the CloudWatch either.

[jayair]

@akolybelnikov So using a console.log before the data is returned does not show anything in the CloudWatch logs?

[akolybelnikov]

Hi @jayair here's what I get in CW:

(92123dc7-bb56-11e7-81fe-9d159db3aeda) Endpoint response body before transformations:
{
    "statusCode": 200,
    "headers": "{\"Access-Control-Allow-Origin\":\"*\",\"Access-Control-Allow-Credentials\":true}",
    "body": "{\"userId\":\"eu-central-1:e393fcdf-4bba-4a36-b844-c8bfafc20fc7\",\"productId\":\"921c76d0-bb56-11e7-8c22-9b7368c0b402\",\"category\":\"bread\",\"productName\":\"Moscower\",\"content\":\"traditional loaf\",\"attachment\":\"loaf.jpg\",\"price\":\"103.98 RUB\",\"createdAt\":1509136615101}"
}

(92123dc7-bb56-11e7-81fe-9d159db3aeda) Endpoint response headers: {x-amzn-Remapped-Content-Length=0, Connection=keep-alive, x-amzn-RequestId=92188007-bb56-11e7-8447-a501c93e3a7a, Content-Length=409, Date=Fri, 27 Oct 2017 20:36:55 GMT, X-Amzn-Trace-Id=root=1-59f398e7-9fc85288e6a24722051885b7;sampled=0, Content-Type=application/json}

(92123dc7-bb56-11e7-81fe-9d159db3aeda) Execution failed due to configuration error: Malformed Lambda proxy response

(92123dc7-bb56-11e7-81fe-9d159db3aeda) Method completed with status: 502

It's 200 followed by 502.

[jayair]

@akolybelnikov If you check the tutorial, the headers need to be a JSON object not a string - https://github.com/AnomalyInnovations/serverless-stack-demo-api/blob/master/libs/response-lib.js#L12

[akolybelnikov]

@jayair I must have changed that line trying to refactor the code,. I have changed the line back to the JSON object, it doesn't have any effect though, still getting the 502. We don't have to re-deploy once we change the code locally, do we?

[akolybelnikov]

issue resolved. I did not re-deploy after local changes. what was I thinking? don't ask me. getting proper response now.

[chaosdarky]

Hi, i’m still getting the error. Hope someone can help to debug it.

[jayair]

@chaosdarky Can you check your Cognito User Pool and Identity Pool to check it there is a user in there? In the User Pool there should be a user with the email admin@example.com.

[chaosdarky]

@jayair there are user there. I was able to authenticate and Identity Pool shows that i have 1 new identity so its working but I don’t know why I am still getting a null CognitoIdendityID

[jayair]

@chaosdarky And this happens even if you tried it as a new user? As in, create another user and try it again?

[toyeebgodo]

I got a 403 response:

Authenticating with User Pool Getting temporary credentials Making API request { status: 403, statusText: 'Forbidden', data: { message: 'Forbidden' } }

Then I followed @bradodarb's idea:

I changed this: --invoke-url='https://xxxx.execute-api.us-west-2.amazonaws.com/prod' --path-template='/notes'

to

--invoke-url='https://xxxx.execute-api.us-west-2.amazonaws.com/prod/notes' --path-template=''

and my API testing worked.

[jayair]

@toyeebgodo Hmmm that's weird. But thanks for reporting!

[tommedema]

Is it possible that "something" (perhaps a serverless deploy or a bug within AWS) somehow changed the policy document of the Auth role associated to the federated identity we created?

For some reason, I suddenly started getting the same error as certain people above: User: arn:aws:sts::904892619278:assumed-role/Cognito_notesidentitypoolAuth_Role/CognitoIdentityCredentials is not authorized to perform: execute-api:Invoke on resource: arn:aws:execute-api:us-west-2:********9278:zgbqusr8ua/prod/POST/notes

When checking the policy document, it seemed like the wildcard was missing:

The thing is, I never removed it myself, and the wildcard certainly used to be there. Restoring it resolves the issue

But the real question is: what could have changed the policy document in the first place?

[jayair]

@tommedema That's really strange...

[lucasgonze]

I am one of those getting a 502, but the details are different. I have read the discussion so far and tried relevant ideas for myself. On inspecting the logs I see this error, which has not yet been discussed:

(c1dc9b55-c4e3-11e7-ba33-6db69d7dccc2) Endpoint response body before transformations:
{
    "errorMessage": "EACCES: permission denied, open '/var/task/create.js'",
    "errorType": "Error",
    "stackTrace": [
        "Object.fs.openSync (fs.js:641:18)",
        "Object.fs.readFileSync (fs.js:509:33)",
        "Object.Module._extensions..js (module.js:578:20)",
        "Module.load (module.js:487:32)",
        "tryModuleLoad (module.js:446:12)",
        "Function.Module._load (module.js:438:3)",
        "Module.require (module.js:497:17)",
        "require (internal/module.js:20:19)"
    ]
}

[jayair]

@lucasgonze That's pretty strange. I haven't seen that before but some Google searching shows that maybe it's related to the file permissions of create.js? Here is some more info:

• https://acloud.guru/forums/aws-lambda/discussion/-KSVv58PhKhA1c6a6EZ-/errormessage-eacces-permission-denied-open-vartaskcsvreadjs
• https://github.com/danilop/LambdAuth/issues/14

[lucasgonze]

@jayair, these links are a useful clue. My permissions turn out to be 600 rather than 644, probably because of the default mask in termux/android. I will debug from this hypothesis.

[lucasgonze]

The EACCESS issue is very likely that permissions on create.js are 660 rather than 644 or 664. That is, only my user account and its unix group have read permission.

660 is an interaction with how zip works in termux. The issue is specific to termux.

This is probably unfixable. Other termux users can track (or vote for) the issue here: https://github.com/termux/termux-packages/issues/1799

The best I can imagine serverless-stack and the Serverless framework doing is to warn termux users.