Lua module for AWS APIs . The missing AWS SDK from Nginx/Openresty. Use it to proxy AWS APIs in a simple fashion, with any Http Client that you prefer.
This library is considered production ready.
It needs a bit of love to support more AWS APIs as the list of APIs is small at the moment.
But even if not all AWS APIs are exposed via Lua class wrappers, you can still use any AWS API via a generic Lua wrapper AwsService
.
This library requires an nginx build with OpenSSL, the ngx_lua module, LuaJIT 2.0, api-gateway-hmac module, and lua-resty-http module.
Requests to AWS Services must supply valid credentials and this library provides a few credentials providers for signing AWS Requests.
aws_credentials
config option specifies which provider to use.
If no aws_credentials
is provided then the library will try to find one using the following order:
aws_access_key
and aws_secret_key
are provided then Basic Credentials Provider is used. INFO: This library supports the latest AWS V4 signature which means you can use any of the latest AWS APIs without any problem.
Basic credentials work with secret_key
and access_key
.
aws_credentials = {
provider = "api-gateway.aws.AWSBasicCredentials",
access_key = "replace-me",
secret_key = "replace-me"
}
INFO: For better security inside the AWS environment use IAM or STS credentials.
This is probably the most popular credentials provider to be used inside the AWS environment. To learn more about IAM Credentials see IAM Roles for Amazon EC2.
This credentials provider discovers automatically the IAM Role
associated to the EC2 instance retrieving its credentials and caching them.
This is a more secure method for signing AWS requests as credentials are short lived and the NGINX configuration doesn't need to maintain any access_key
or secret_key
nor worry about rotating the keys.
aws_credentials = {
provider = "api-gateway.aws.AWSIAMCredentials",
shared_cache_dict = "my_dict" -- the name of a shared dictionary used for caching IAM Credentials
}
AWS Security Token Service(STS) provides a great way to get limited-privilege credentials for accessing AWS Services. To learn more about STS Credentials see Getting Temporary Credentials with STS and the STS section bellow.
aws_credentials = {
provider = "api-gateway.aws.AWSSTSCredentials",
role_ARN = "arn:aws:iam::111111:role/assumed_role_kinesis", -- ARN of the role to assume
role_session_name = "kinesis-session", -- a name for this session
shared_cache_dict = "shared_cache" -- shared dict for caching the credentials
}
Unlike IAM Credentials that exposes a single IAM Role for each EC2 instance, STS Credentials allows an EC2 instance to assume multiple roles each with its own access policy.
This credentials provider uses SecurityTokenService for making requests to STS and SecurityTokenService uses the IAM Credentials provider for making the call.
It is strongly recommended to provide the shared_cache_dict
in order to improve performance. The temporary credentials obtained from STS are stored in the shared_cache_dict
for up to 60 minutes.
AwsService is a generic Lua class to interact with any AWS API. The actual implementations extend this class. Its configuration is straight forward:
local service = AwsService:new({
aws_service = "sns",
aws_region = "us-east-1",
aws_credentials = {
provider = "api-gateway.aws.AWSIAMCredentials",
shared_cache_dict = "my_dict" -- the name of a shared dictionary used for caching IAM Credentials
}
aws_debug = true, -- print warn level messages on the nginx logs. useful for debugging
aws_conn_keepalive = 60000, -- how long to keep the sockets used for AWS open
aws_conn_pool = 100 -- the connection pool size for sockets used to connect to AWS
})
local KinesisService = require "api-gateway.aws.kinesis.KinesisService"
local service = KinesisService:new({
aws_region = ngx.var.aws_region,
aws_secret_key = ngx.var.aws_secret_key,
aws_access_key = ngx.var.aws_access_key
})
-- CreateStream
local response = service:createStream("test-stream")
-- ListStreams
local streams = service:listStreams()
-- PutRecord
local response = service:putRecord("test-stream","test-message", "partitionKey")
-- PutRecords
local records = {
{
Data = "test-data-1",
PartitionKey = "partitionKey-1"
},
{
Data = "test-data-2",
PartitionKey = "partitionKey-2"
}
}
local response = service:putRecords("test-stream", records)
local LambdaService = require "api-gateway.aws.lambda.LambdaService"
local service = LambdaService:new({
aws_region = ngx.var.aws_region,
aws_debug = true, -- print warn level messages on the nginx logs
aws_conn_keepalive = 60000, -- how long to keep the sockets used for AWS alive
aws_conn_pool = 100 -- the connection pool size for sockets used to connect to AWS
})
--Invoke function
local payload = {
key1 = "value-1",
key2 = "value-2"
}
local functionName = "hello-world-test"
local invokeResult, code, headers, status, body = service:invoke(functionName, payload)
ngx.say("EXECUTION RESULT:" .. tostring(body))
--Invoke a Lambda function from another AWS account
functionName = "arn:aws:lambda:us-east-1:123123123123123:function:hello-world"
invokeResult, code, headers, status, body = service:invoke(functionName, payload)
ngx.say("EXECUTION RESULT FROM ANOTHER AWS ACCOUNT:" .. tostring(body))
Note that in order to call a Lambda function cross AWS Accounts you need to have the correct policies in place.
Let's say an EC2 node in the account 789789789789789
with a role webserver
needs to call a lambda function defined in another AWS account 123123123123123
.
Create a new Lambda function in account 123123123123123
, i.e hello-world-lambda-fn
Make sure that the webserver
role in account 789789789789789
can call Lambda functions from other AWS accounts.
Create a new role policy for webserver
:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1437070759000",
"Effect": "Allow",
"Action": [
"lambda:InvokeFunction"
],
"Resource": [
"arn:aws:lambda:*:*:*:*"
]
}
]
}
123123123123123
where you want to invoke the Lambda function and using AWS CLI add a new policy for your lambda function$ aws lambda add-permission \
--function-name hello-world-lambda-fn \
--statement-id stmt-lambda-Id-456 \
--action "lambda:InvokeFunction" \
--principal arn:aws:iam::789789789789789:role/webserver
--region us-east-1
# should yield
{
"Statement": "{\"Action\":[\"lambda:InvokeFunction\"],\"Resource\":\"arn:aws:lambda:us-east-1:123123123123123:function:hello-world-lambda-fn\",\"Effect\":\"Allow\",\"Principal\":{\"AWS\":\"arn:aws:iam::789789789789789:role/webserver\"},\"Sid\":\"stmt-lambda-Id-456\"}"
}
You have to make sure that the user you're adding the permissions with does have the rights to lambda:AddPermission
and lambda:GetPolicy
.
To verify you have the policy added you can execute:
aws lambda get-policy --function-name hello-world-lambda-fn --region=us-east-1
local SnsService = require "api-gateway.aws.sns.SnsService"
local service = SnsService:new({
aws_region = ngx.var.aws_region,
aws_secret_key = ngx.var.aws_secret_key,
aws_access_key = ngx.var.aws_access_key
})
-- ListTopics
local list = service:listTopics()
local topicArn = list.ListTopicsResponse.ListTopicsResult.Topics[1].TopicArn
-- Publish
local response = service:publish("test-subject","test-message", topicArn)
local messageId = response.PublishResponse.PublishResult.MessageId
local KmsService = require "api-gateway.aws.kms.KmsService"
local service = KmsService:new({
aws_region = ngx.var.aws_region,
aws_secret_key = ngx.var.aws_secret_key,
aws_access_key = ngx.var.aws_access_key
})
-- search for aliases
local list = service:listAliases()
-- pick the first alias
local KeyId = list.Aliases[1].AliasName
-- generate a data key
local cipher = service:generateDataKey(KeyId, "AES_256")
local blob = cipher.CiphertextBlob
local blob_text = cipher.Plaintext
-- encrypt a text
local encryptResult = service:encrypt(KeyId, blob_text)
-- decrypt
local decryptResult = service:decrypt(encryptResult.CiphertextBlob)
The AWS Security Token Service (STS) provides access to temporary, limited-privilege credentials for AWS Identity and IAM users. This can be useful for communicating with a a third party AWS account, without having access to some long-term credentials. (ex. IAM user's access key).
The SecurityTokenService is a AWS STS API wrapper and it provides support for the AssumeRole
requests. It can be used as follows:
local SecuriyTokenService = require "api-gateway.aws.sts.SecuriyTokenService"
local service = SecuriyTokenService:new({
aws_region = ngx.var.aws_region,
aws_secret_key = ngx.var.aws_secret_key,
aws_access_key = ngx.var.aws_access_key
})
local response, code, headers, status, body = sts:assumeRole(role_ARN,
role_session_name,
policy,
security_credentials_timeout,
external_id)
These are the steps that need to be followed in order to be able to generate temporary credentials:
Let's say that the AWS account A
needs to send records to a Kinesis stream in account B
.
Create a role in account B
that grants permission to write to Kinesis and update the Trust Relationship
as follows:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::{A-account-number}:root"
},
"Action": "sts:AssumeRole"
}
]
}
The role from account A
should be set to allow sts:AssumeRole
actions pointing to the B
account:
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Resource": "arn:aws:iam::{B-account-number}:role/*"
}]
}
Call AWS STS AssumeRole API to obtain temporary credentials.
local response, code, headers, status, body = sts:assumeRole(role_ARN,
role_session_name,
policy,
security_credentials_timeout,
external_id)
INFO: For more information on how to configure the accounts see How to Use an External ID When Granting Access to Your AWS Resources to a Third Party.
Since this module is running inside the api-gateway
, make sure the api-gateway binary is installed under /usr/local/sbin
.
You should have 2 binaries in there: api-gateway
and nginx
, the latter being only a symbolik link.
git submodule update --init --recursive
The tests are based on the test-nginx
library.
This library is added a git submodule under test/resources/test-nginx/
folder, from https://github.com/agentzh/test-nginx
.
Test files are located in test/perl
.
The other libraries such as Redis
, test-nginx
are located in test/resources/
.
Other files used when running the test are also located in test/resources
.
To execute the test issue the following command:
TEST_NGINX_AWS_CLIENT_ID="--change--me" TEST_NGINX_AWS_SECRET="--change-me--" make test
If you want to run a single test, the following command helps:
TEST_NGINX_AWS_CLIENT_ID="--change--me" TEST_NGINX_AWS_SECRET="--change-me--" \
PATH=/usr/local/sbin:$PATH TEST_NGINX_SERVROOT=`pwd`/target/servroot TEST_NGINX_PORT=1989 prove -I ./test/resources/test-nginx/lib -r ./test/perl/awsv4signature.t
This command only executes the test awsv4signature.t
.
sudo LUA_LIB_DIR=/usr/local/api-gateway/lualib make install