search

adobe-apiplatform / api-gateway-aws

AWS SDK for NGINX with Lua
Apache License 2.0
171 stars 44 forks source link

use with dynamodb always return 404 #14

Closed jasonbigl closed 7 years ago

jasonbigl commented 7 years ago

Hi guys,

Thank you very much for such a great lib, I'm using it in one of my project, all works fine, except dynamodb, when I try to integrate with dynamodb, I always got 404, do you have any idea about it?

I have also ask aws support and they seems didn't find anything...

here is my code:

local uuid = require 'resty.string.uuid'
local dynamodbService = require 'aws.dynamodb.DynamodbService'

local camp_id = uuid:generate()

local arr = {
    TableName = "movies",
    Item = {
        id = {
            S = camp_id
        },
        name = {
            S = "demo movie"
        }
    },
    ConditionExpression = "id <> :i",
    ExpressionAttributeValues = {}
}

arr["ExpressionAttributeValues"][":i"] = {
    S = camp_id
}

local aws_conn = GVAL['aws_conn']

local service = dynamodbService:new(aws_conn)

local ok, code, headers, status, body = service:putItem(arr)

ngx.say(body or "")

and here is the DynamodbService:

local AwsService = require "aws.AwsService"

local _M = AwsService:new({ ___super = true })
local super = {
    instance = _M,
    constructor = _M.constructor
}

function _M.new(self, o)
    local o = o or {}
    o.aws_service = "dynamodb"

    -- aws_service_name is used in the X-Amz-Target Header, format: DynamoDB_<API version>
    -- http://docs.aws.amazon.com/zh_cn/amazondynamodb/latest/APIReference/Welcome.html "API Reference (API Version 2012-08-10)"
    o.aws_service_name = "DynamoDB_20120810"

    super.constructor(_M, o)

    setmetatable(o, self)
    self.__index = self
    return o
end

-- 
-- API: hhttp://docs.aws.amazon.com/zh_cn/amazondynamodb/latest/APIReference/API_PutItem.html
function _M:putItem(params)
    assert(params ~= nil, "Please provide params.")

    local ok, code, headers, status, body = self:performAction("PutItem", params, "/", "POST", true)

    if (code == ngx.HTTP_OK and body ~= nil) then
        return {}, code, headers, status, body
    end

    return nil, code, headers, status, body
end

return _M

Thank you very much :)

ddragosd commented 7 years ago

Hi @mrleex thanks for the feedback. I need to try it myself too and see more about the 404. Can you share the entire response from AWS in the meanwhile ?

jasonbigl commented 7 years ago

Hi @ddragosd , thank you very much for your reply, sure, this is what I always get:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
  <title>Page Not Found</title>
</head>
<body>Page Not Found</body>
</html>

As you can see, no useful infos, and this is my request header:

POST / HTTP/1.1
Content-Length: 209
User-Agent: lua-resty-http/0.09 (Lua) ngx_lua/10006
Host: dynamodb.us-west-1.amazonaws.com
Accept: application/json
X-Amz-Target: DynamoDB_20120810.PutItem
Content-Type: application/x-amz-json-1.1
Authorization: AWS4-HMAC-SHA256 Credential=xxxxxxxxxxxxxx/20160926/us-west-1/dynamodb/aws4_request,SignedHeaders=host;x-amz-date,Signature=91c93ea50856ca4987ead92e2e7996c7450ee7e76199a47ddbc30748b5f6f7cb
X-Amz-Date: 20160925T025017Z

I have replaced my real secret key with "xxxxxx"( I use the real key when doing test),

and here is my request body:

{"ExpressionAttributeValues":{":i":{"S":"d3101db5b5574342962fb83f82a6e02b"}},"Item":{"name":{"S":"demo movie"},"id":{"S":"d3101db5b5574342962fb83f82a6e02b"}},"TableName":"movies","ConditionExpression":"id <> :i"}

I'm very sure the table is exist and active in the us-west-1 zone.

However, your lib works great with kinesis, this is what confused me....

Thank you.

jasonbigl commented 7 years ago

Hi @ddragosd , any updates, this really blow up my head...thank you in advance

ddragosd commented 7 years ago

@mrleex I have this in my TODOs, but I didn't manage to get to work on it yet.

I can share a tip, in case it helps you in the meanwhile: when I find myself in this situation I usually use the AWS CLI - the python script - and make the request to the AWS Service using the --debug flag. Then I look at the HTTP request and compare it against what NGINX would send. If you get the chance to do this, it would be super helpful to share the output of the python script.

jasonbigl commented 7 years ago

@ddragosd Thank you very much, I have used aws official php sdk which worked with dynamodb, and I have compared the php sdk http request and ngx lua http request, they are the same....

Anyway, I will also try your way, your time is really appreciated :)

ddragosd commented 7 years ago

@mrleex I've noticed a difference when comparing with the AWS DOCS: Content-Type: application/x-amz-json-1.0 vs Content-Type: application/x-amz-json-1.1. We can't be sure that this matters until we get to compare a successful request and an unsuccessful one. If this matters then we can overwrite the content type in the performAction method:

self:performAction("PutItem", params, "/", "POST", true, 60000, "application/x-amz-json-1.0")
ddragosd commented 7 years ago

It looks like the Content-Type is indeed application/x-amz-json-1.0

My successful request with the python CLI shows the following:

2016-10-11 19:55:53,928 - MainThread - botocore.auth - DEBUG - CanonicalRequest:
POST
/

content-type:application/x-amz-json-1.0
host:dynamodb.us-east-1.amazonaws.com
x-amz-date:20161012T025553Z
x-amz-security-token:**********
x-amz-target:DynamoDB_20120810.PutItem

content-type;host;x-amz-date;x-amz-security-token;x-amz-target

Can you try my suggestion above and confirm back the result ?

jasonbigl commented 7 years ago

@ddragosd Oh...I apologize for the delayed reply, I tried as you suggested and it works!!!!

Oh, man, you rocks! I didn't notice the json-1.0 and json-1.1....so to make the dynamodb works, we need

self:performAction("PutItem", params, "/", "POST", true, 60000, "application/x-amz-json-1.0")

and we also need to modify the AwsService.lua, change the function performAction, add the following:

if content_type == "application/x-amz-json-1.1" or content_type == "application/x-amz-json-1.0" then
        request_body = cjson.encode(arguments)

really appreciate your time and help :)

ddragosd commented 7 years ago

@mrleex I'm glad we found the issue; thanks for the confirmation. Feel free to contribute the DynamodbService in a pull request. Others might find it useful as well.