AWS related

[changtimwu]

my first time using elastic beanstalk(EB). I'm like to use EB as the gateway to bridge OverC and AWS IoT.

It default turns on ELB and ELB has some issues about websocket.

http://stackoverflow.com/questions/9184895/how-do-you-get-amazons-elb-with-https-ssl-to-work-with-web-sockets

[changtimwu]

using route 53 to archive global service http://www.cyberciti.biz/cloud-computing/aws/route-53-geodns-tutorial/

dynamodb is good at self-hosting but it has many limitation when compared with mongodb. kinesis https://www.quora.com/What-can-you-do-with-Amazon-Kinesis

good points on aws iot and nice integration api-gateway, lambda, dynamodb https://community.particle.io/t/mqtt-and-new-aws-iot-service/16483/13

[changtimwu]

aws iot quickstart

• developer guide. Good reference.
• make sure your awscli has iot command

aws iot help

if not please install the lastest formulat

brew install awscli --HEAD

[changtimwu]

aws IAM is so complicated.

• 從這裡開始, 選 Security Credentials
• aws 舊有的帳號管理系統只有單一 Security Credentials , 只有單一帳號密碼登錄, api 就是用那把單一的 access key/secret, 後來 AWS 開始建議 IAM best practice , 重點就是避免單一root帳號擁有極大權限卻只用密碼保護.
• 帳號登錄開始支援 MFA, 有點像簡訊輸入authentication code, 它是用 google authenticator 這個app, 這樣就比密碼保護安全很多.
• IAM 觀念上大致是 ACL, 項目有 user, group, role, policy

user & group 的觀念

• 可以開多個 user, 每個 user 的 permission 欄位可以 attach 多個 policies, 代表他被允許的動作
• 每個 user 的識別字串大概是這樣, arn:aws:iam::406995953077:user/timwu
• 每個 user 可以有自己的 access key and secret, 同樣放在 ~/.aws/credentials, awscli 能下的指令就會被 policy 限制
• 每個 user 可以各自設 MFA(multi-factor authentication), MFA device 識別字串長這樣 arn:aws:iam::406995953077:mfa/iotadmin
• 可以把開 group, group 也有 permission, 也是attach policies, 把 user 加到 group中, 就會以同樣的policy生效.

policy的觀念

• 來看一個典型的 policy document

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "iot:Connect",                "iot:Publish",                "iot:Subscribe",                "iot:Receive",                "iot:GetThingShadow",                "iot:UpdateThingShadow"
            ],
            "Resource": "*"
       }
    ]
}

• 可以做什麼 最細的單位叫做 action, 跟 SDK 裡的 API 或 awscli subcommand 幾乎是一對一
• policy 可以綁到 entity 上, 而entity 就是 use or role.
• aws console 那一百多個 policy 是先建好的, 稱為 AWS managed policy, 可以自己再加. 稱為 customer managed policy

另一個 policy

{
  "Version": "2012-10-17",
  "Statement": {
    "Effect": "Allow",
    "Action": ["s3:GetObject"],
    "Resource": "arn:aws:s3:::Hello-bucket/*",
    "Condition": {"StringEquals": {"s3:prefix": "Bobs-"}}
}

可以理解成 Allow or not to do Action on Resource when Condition Resource 就是 AWS 所有服務的operation 都會有 ARN 表示法 官方文件

role 的觀念

• user/group policy 的觀念看似不錯, 但其實都得帶著 credentials (key & secret), 如何 distribute 是個難題, 舉個例, 開多個 EC2 instance, 上面跑的 code 想取用 aws service(ex. S3), 那麼就得把 credential copy進去每個 instance.
• 官方解釋 it's challenging to securely distribute credentials to each instance, especially those that AWS creates on your behalf, such as Spot Instances or instances in Auto Scaling groups. You must also be able to update the credentials on each instance when you rotate your AWS credentials.
• role 一樣是 attach 某些 policy
• role 就是在某些aws service 下(ex. ec2 ), 讓你可以指定 role , 有了 role就只看role的policy , 不需要管 user/group. 所以開 ec2時在 profile都可以指定 role. 帶著某個 role 開的 instance 就會自動有那個role的權限.
• 原文描述 Also, a role does not have standard long-term credentials such as a password or access keys associated with it. Instead, when you assume a role, it provides you with temporary security credentials for your role session.
• 基本範例 https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html
• role 有個屬性叫 trust relationship, 就是嚴格定義只有某些 identity (ex. EC2) 能套用此 role.

"Principal": {
   "Service": "ec2.amazonaws.com"
 }

• 不只 EC2, 只要是那個服務需要取用其他服務, 通常都會有 role 可以指定. ex. lambda
• 官方文件

role 的 security model

• 沒有 role 的話, 本來是很單純的

  user -> group -> group policies -> actions

• 加了 role 變成

  user -> group ->  group policies -> actions -> 
  action triggers some resource -> resource has a role->  role policies-> actions

  比如

  aws lambda list-functions

  lambda function 簡介

  aws lambda get-function --function-name xxx

  查 function policy

  aws lambda get-policy --output text --function-name xxx

查 function role

 aws iam get-role --role-name  xxxrole

[changtimwu]

EC2的 key pair 是 ssh 用的, 跟 aws credentials(key & secret) 是不一樣的

[changtimwu]

https://github.com/jaws-framework/JAWS looks interesting. Serverless cloud. https://www.youtube.com/watch?v=D_U6luQ6I90&feature=youtu.be https://aws.amazon.com/tw/blogs/compute/aws-lambda-sessions-at-reinvent-2015-wrap-up/

[changtimwu]

got such error when entering aws iot-data

/usr/local/Cellar/awscli/1.9.6/libexec/vendor/lib/python2.7/site-packages/botocore/handlers.py:488:
 UnsupportedTLSVersionWarning: Currently installed openssl version: OpenSSL 0.9.8zg 14 July 2015 
does not support TLS 1.2, which is required for use of iot-data. Please use python installed with openssl 
version 1.0.1 or higher.  UnsupportedTLSVersionWarning

firstly upgrade openssl. The following commands install openssl/libssl/libcrypto into /usr/local/Cellar

brew install openssl
brew link --force openssl

check if the version is 1.0.x at new shell

openssl version

problem still there.

dissection

• dependency: awscli -> python -> ssl package -> libssl
• brew installs openssl but it can't replace OSX's libssl/libcrypto in /usr/lib when you manually do so, system complains about permission error
• We can't reinstall ssl package with pip install --upgrade ssl since ssl is becomes part of python distribution
• brew install python makes progress. Check the newly installed python's ssl version

import ssl
print ssl.OPENSSL_VERSION_INFO

• awscli is implemented with python. Its main command /usr/local/Cellar/awscli/1.9.6/libexec/bin/aws harecode #!/usr/bin/python, which would use OSX builtin python. so modify it from #!/usr/bin/python to #!/usr/bin/env/python to use brew python.

AWS shell

• https://github.com/awslabs/aws-shell
• much user-friendly interface. colorful options and autocomplete!

[changtimwu]

https://snowulf.com/2015/08/05/tutorial-aws-api-gateway-to-lambda-to-dynamodb/ go through from dynamodb, lambda, api gateway in 10 steps

[changtimwu]

RDS security group 跟 VPC security group 不一樣 http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Overview.RDSSecurityGroups.html

• Three types of security groups are used with Amazon RDS: DB security groups: In simple terms, a DB security group controls access to a DB instance that is not in a VPC. 從外面存取. 用 CIDR 過濾.
• VPC security groups: A VPC security group controls access to a DB instance (or other AWS instances) inside a VPC.
• EC2 security groups: An EC2 security group controls access to an EC2 instance.( 發現最後生效的是這個), DB security group 反而沒作用?

[changtimwu]

~/.aws/credentials 記錄每個 profile 的 key id and secret profile 不等於 AWS IAM 裡的 user account, 它還加上了 region, ex. 可以創造 3 profile

• alice_useast
• alice_apse
• bob

在 ~/.aws/config 裡面設定 region

[changtimwu]

common commands for aws or aws-shell

s3 ls
iam list-users
iam list-groups
iam list-roles

query role policy

iam get-role --role-name  serverless-hello-world-dev-us-east-1-lambdaRole
iam list-role-policies  --role-name serverless-hello-world-dev-us-east-1-lambdaRole
iam get-role-policy --role-name  serverless-hello-world-dev-us-east-1-lambdaRole  --policy-name dev-serverless-hello-w

[changtimwu]

list allow roles' condition & service

aws iam list-roles | jq '.Roles[] | 
{ 
  RoleName: .RoleName, 
  Description: .Description, 
  Service: .AssumeRolePolicyDocument.Statement[].Principal.Service, 
  Condition: .AssumeRolePolicyDocument.Statement[].Condition
}

role 也有 condition

{
  "RoleName": "iotex_smsverification_MOBILEHUB_1697124871",
  "Description": null,
  "Service": "cognito-idp.amazonaws.com",
  "Condition": {
    "StringEquals": {
      "sts:ExternalId": "dc65d449-6569-4fc9-8ff9-cd2c9adce052"
    }
  }
}