search

changtimwu / changtimwu.github.com

Tim's testing/practice notes
7 stars 2 forks source link

AWS devops professional #95

Open changtimwu opened 6 years ago

changtimwu commented 6 years ago

devops professional certification

https://aws.amazon.com/certification/certified-devops-engineer-professional/

white papers

advices

changtimwu commented 6 years ago

https://aws.amazon.com/datapipeline/

changtimwu commented 6 years ago

elasticbeanstalk

https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features.deploy-existing-version.html https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features.rolling-version-deploy.html https://www.quora.com/Whats-the-difference-between-immutable-and-blue-green-deployments-on-AWS-Elastic-Beanstalk https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_docker_ecs.html https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features.CNAMESwap.html

cloudformation

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-validate-template.html https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-custom-resources.html https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/create-reusable-transform-function-snippets-and-add-to-your-template-with-aws-include-transform.html http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/updating.stacks.walkthrough.html https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/best-practices.html https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-custom-resources.html

ELB & autoscaling

https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-cloudwatch-metrics.html https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-healthchecks.html https://medium.com/qbits/autoscaling-using-custom-metrics-5f977903bc45 https://docs.aws.amazon.com/autoscaling/ec2/userguide/LaunchConfiguration.html http://docs.aws.amazon.com/autoscaling/latest/userguide/as-enter-exit-standby.html https://docs.aws.amazon.com/autoscaling/ec2/userguide/Cooldown.html https://docs.aws.amazon.com/autoscaling/ec2/userguide/as-suspend-resume-processes.html#as-suspend-resume https://docs.aws.amazon.com/autoscaling/ec2/userguide/AutoScalingGroupLifecycle.html

monitoring

https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/WhatIsCloudWatchLogs.html https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-log-file-validation-intro.html https://aws.amazon.com/about-aws/whats-new/2016/03/cloudwatch-events-now-supports-amazon-sqs-queue-targets/ https://docs.aws.amazon.com/appsync/latest/devguide/monitoring.html

OPSWORKS

https://aws.amazon.com/opsworks/stacks/ https://aws.amazon.com/blogs/aws/using-aws-elasticache-for-redis-with-aws-opsworks/

CI/CD

https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/dns-failover-determining-health-of-endpoints.html https://docs.aws.amazon.com/codedeploy/latest/userguide/deployment-steps.html https://aws.amazon.com/premiumsupport/knowledge-center/auto-scaling-group-rolling-updates/

IAM

https://docs.aws.amazon.com/IAM/latest/UserGuide/tutorial_cross-account-with-roles.html https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_getting-report.html

MISC

minor

https://docs.aws.amazon.com/directoryservice/latest/admin-guide/directory_ad_connector.html https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSEncryption.html

changtimwu commented 6 years ago

rolling update

immutable update

* always create new instances

一個 eb application 可以有多個 env, 想像就是用同一份 application source, 建立出每不同的 deploy 參數,
instance & URL 都會不一樣.

所以 blue-green deploy, 就是用上面這招, 現有在跑的是 blue env, 把新版用green env deploy 出去, 然後進行互換

Rolling Updates versus Rolling Deployments

changtimwu commented 6 years ago

cloudformation

basics

basic form of a CF entry

NameOfStuff:
  Type: AWS::tellFormatOfStuff
  Properties:
     stuffContent1:
     stuffContent2:
     ....

It's obvious that a CF parser basically a program travels a tree and process keys named Type. reference

transform

to include a CF clip in S3

Transform:
  Name: 'AWS::Include'
  Parameters:
    Location: 's3://MyAmazonS3BucketName/MyFileName.yaml'

the AWS::Serverless Transform extends resource types

  MyServerlessFunctionLogicalID:
    Type: AWS::Serverless::Function
    Properties:
      Handler: index.handler
      Runtime: nodejs4.3
      CodeUri: 's3://testBucket/mySourceCode.zi
changtimwu commented 6 years ago

lambda blue/green deployment

https://docs.aws.amazon.com/codestar/latest/userguide/how-to-modify-serverless-project.html something like rolling deployment for lambda

簡單的說 lambda function 更新了以後, traffic 仍然只導到舊的 lambda function, 慢慢根據底下策略的不同, 把 traffic 移到慢慢移到新的新的 lambda function codedeploy 上面有選項

用 codestar gen 出來的 lambda project, template.yml 會有下列, 想要 blue-green deploy, 就把 comment 拿掉, 讓它生效

Globals:
  Function:
    AutoPublishAlias: live
    DeploymentPreference:
      Enabled: true
      Type: Canary10Percent5Minutes
changtimwu commented 6 years ago

codestar

其實 codestar 就是大 setup wizard + dashboard, 包含 codepipeline 的 wizard.

EC2 based App

beanstalk based App

lambda based app

Insights

changtimwu commented 6 years ago

codestar 很方便, 但要搞清楚 local git push 後發生哪些事

codepipeline

basics

stage

action

deploy stage

changtimwu commented 6 years ago

CLI common options

changtimwu commented 6 years ago

AWS ML

summarized in https://aws.amazon.com/tw/machine-learning/

NLP

Image & Video

Data Analysis