pahud / ecs-cfn-refarch

Amazon ECS reference architecture
102 stars 62 forks source link

Confusion #4

Closed sibs786 closed 5 years ago

sibs786 commented 5 years ago

Dear,

I am trying to fetch ecs mixed instance policy

1) ECS Mixed Instance Policy: This is somewhat i feel is important Outputs: instanceProfile: Description: Instance profile ARN Value: Fn::GetAtt:

I am confused about the IAM Roles It needs. I have removed some values from it which are clear to me. Do we need lambda function for this ? Is it important? Any Work Required on ecs instance to make it work?

pahud commented 5 years ago

Hi @sibs786 I don't understand your question.

The primary cloudformation template is here https://github.com/pahud/ecs-cfn-refarch/blob/master/cloudformation/service.yaml

Can you indicate which lines you were asking about?

sibs786 commented 5 years ago

Hi pahud thanks for replying. I have ignored the above configuration.

This is my autoscaling group configuration now ECSAutoScalingGroup: DependsOn: ECSCluster Type: AWS::AutoScaling::AutoScalingGroup Properties: MixedInstancesPolicy: InstancesDistribution: OnDemandAllocationStrategy: prioritized OnDemandBaseCapacity: !Ref OnDemandBaseCapacity OnDemandPercentageAboveBaseCapacity: !Ref OnDemandPercentageAboveBaseCapacity SpotAllocationStrategy: lowest-price SpotInstancePools: !Ref SpotInstancePools LaunchTemplate: LaunchTemplateSpecification: LaunchTemplateId: !Ref MyLaunchTemplate
VPCZoneIdentifier: !Ref Subnets MinSize: 2 MaxSize: 5 DesiredCapacity: 2 Tags:

Need to understand why using this ECSTaskRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: '2012-10-17' Statement:

Your LaunchTempalate MyLaunchTemplate: Type: AWS::EC2::LaunchTemplate Properties: LaunchTemplateName: !Sub "eksLaunchTemplate-${AWS::StackName}" LaunchTemplateData: TagSpecifications:

        ResourceType: instance
        Tags:
          - Key: Name
            Value: !Sub "${ECSDefaultCluster}-ASG-Node"

    UserData: 
      "Fn::Base64": 
        !Sub |
          Content-Type: multipart/mixed; boundary="==BOUNDARY=="
          MIME-Version: 1.0

          --==BOUNDARY==
          Content-Type: text/x-shellscript; charset="us-ascii"
          #!/bin/bash
          yum install -y awscli

          echo ECS_CLUSTER=${AWS::StackName}-ecs-cluster >> /etc/ecs/ecs.config
          echo ECS_DISABLE_IMAGE_CLEANUP=false >> /etc/ecs/ecs.config

          iid=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)
          export AWS_DEFAULT_REGION=${AWS::Region}
          ilc=$(aws ec2 describe-instances --instance-ids  $iid  --query 'Reservations[0].Instances[0].InstanceLifecycle' --output text)
          if [ "$ilc" == "spot" ]; then
            echo ECS_INSTANCE_ATTRIBUTES='{"instance-purchase-option":"spot"}' >> /etc/ecs/ecs.config
          else
            echo ECS_INSTANCE_ATTRIBUTES='{"instance-purchase-option":"ondemand"}' >> /etc/ecs/ecs.config
          fi

          yum update -y
          # Install awslogs and the jq JSON parser
          yum install -y awslogs jq aws-cfn-bootstrap

          # Inject the CloudWatch Logs configuration file contents
          cat > /etc/awslogs/awslogs.conf <<- EOF
          [general]
          state_file = /var/lib/awslogs/agent-state        

          [/var/log/dmesg]
          file = /var/log/dmesg
          log_group_name = /aws/ECS/var/log/dmesg
          log_stream_name = {cluster}/{container_instance_id}

          [/var/log/messages]
          file = /var/log/messages
          log_group_name = /aws/ECS/var/log/messages
          log_stream_name = {cluster}/{container_instance_id}
          datetime_format = %b %d %H:%M:%S

          [/var/log/docker]
          file = /var/log/docker
          log_group_name = /aws/ECS/var/log/docker
          log_stream_name = {cluster}/{container_instance_id}
          datetime_format = %Y-%m-%dT%H:%M:%S.%f

          [/var/log/ecs/ecs-init.log]
          file = /var/log/ecs/ecs-init.log.*
          log_group_name = /aws/ECS/var/log/ecs/ecs-init.log
          log_stream_name = {cluster}/{container_instance_id}
          datetime_format = %Y-%m-%dT%H:%M:%SZ

          [/var/log/ecs/ecs-agent.log]
          file = /var/log/ecs/ecs-agent.log.*
          log_group_name = /aws/ECS/var/log/ecs/ecs-agent.log
          log_stream_name = {cluster}/{container_instance_id}
          datetime_format = %Y-%m-%dT%H:%M:%SZ

          [/var/log/ecs/audit.log]
          file = /var/log/ecs/audit.log.*
          log_group_name = /aws/ECS/var/log/ecs/audit.log
          log_stream_name = {cluster}/{container_instance_id}
          datetime_format = %Y-%m-%dT%H:%M:%SZ

          EOF

          --==BOUNDARY==
          Content-Type: text/x-shellscript; charset="us-ascii"
          #!/bin/bash
          # Set the region to send CloudWatch Logs data to (the region where the container instance is located)
          region=$(curl -s 169.254.169.254/latest/dynamic/instance-identity/document | jq -r .region)
          sed -i -e "s/region = us-east-1/region = $region/g" /etc/awslogs/awscli.conf

          start ecs
          /opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackName} --resource ASGDefault --region ${AWS::Region} 

          --==BOUNDARY==
          Content-Type: text/upstart-job; charset="us-ascii"

          #upstart-job
          description "Configure and start CloudWatch Logs agent on Amazon ECS container instance"
          author "Amazon Web Services"
          start on started ecs

          script
            exec 2>>/var/log/ecs/cloudwatch-logs-start.log
            set -x

            until curl -s http://localhost:51678/v1/metadata
            do
              sleep 1   
            done

            # Grab the cluster and container instance ARN from instance metadata
            cluster=$(curl -s http://localhost:51678/v1/metadata | jq -r '. | .Cluster')
            container_instance_id=$(curl -s http://localhost:51678/v1/metadata | jq -r '. | .ContainerInstanceArn' | awk -F/ '{print $2}' )

            # Replace the cluster name and container instance ID placeholders with the actual values
            sed -i -e "s/{cluster}/$cluster/g" /etc/awslogs/awslogs.conf
            sed -i -e "s/{container_instance_id}/$container_instance_id/g" /etc/awslogs/awslogs.conf

            service awslogs start
            chkconfig awslogs on
          end script
          --==BOUNDARY==--

    IamInstanceProfile: 
      Arn: !GetAtt EC2InstanceProfile.Arn
    KeyName: !Ref SshKeyName
    NetworkInterfaces: 
      - 
        DeviceIndex: 0
        AssociatePublicIpAddress:
          !If
            - IsASGAutoAssignPublicIp
            - 'true'
            - 'false'
        SubnetId: !Select [0, !Ref SubnetIds]
        Groups: 
          - !Ref NodeSecurityGroup
    ImageId: !Ref ECSAMI

I can see you have combined roles here, also the user data.

This is the launchconfiguration i was using ECSLaunchConfiguration: Type: AWS::AutoScaling::LaunchConfiguration Properties: ImageId: !Ref ECSAMI InstanceType: !Ref InstanceType SecurityGroups:

The Roles i am using here is this ECSRole: Type: AWS::IAM::Role Properties: Path: / RoleName: !Sub ${EnvironmentName}-ECSRole-${AWS::Region} AssumeRolePolicyDocument: | { "Statement": [{ "Action": "sts:AssumeRole", "Effect": "Allow", "Principal": { "Service": "ec2.amazonaws.com" } }] } ManagedPolicyArns:

Can you please guide me how to adjust IAM Roles and replace launch configuration with launch template. I would really appreciate that

sibs786 commented 5 years ago

Ok as per my understanding: You have assigned taskrole and taskexecutionrole to taskdefination which is required. Also the parameters that you have defined in mylaunchtemplate are somewhat similar to my launch configuration

Is that correct?.

pahud commented 5 years ago

https://github.com/pahud/ecs-cfn-refarch/blob/09433d07b62cc711cce7cbe91487eecf123b0c7a/cloudformation/service.yaml#L506-L542

ECSTaskExecutionRole is primarily for ssm:GetParameters and ECSTaskRole is for application running in the Task.

Previously, ssm:GetParameters is only possible within ECS task, but now, because ECS already have native integration with Parameter Store, we can let ECS Execution Role do ssm:GetParameters for us.

https://github.com/pahud/ecs-cfn-refarch/blob/09433d07b62cc711cce7cbe91487eecf123b0c7a/cloudformation/service.yaml#L976-L982

https://github.com/pahud/ecs-cfn-refarch/blob/09433d07b62cc711cce7cbe91487eecf123b0c7a/cloudformation/service.yaml#L467-L503

So basically, in my reference, because Caddy doesn't need to do any AWS API call, we can disable the ECSTaskRole in this case. However, if your task need to call AWS SDK, you will need an appropriate ECSTaskRole with appropriate privileges on it.

pahud commented 5 years ago

Launch Config and Launch Template are totally different. I suggest you refer to my reference about ASG with LT and see how ASG refers to the LT.

https://github.com/pahud/ecs-cfn-refarch/blob/09433d07b62cc711cce7cbe91487eecf123b0c7a/cloudformation/service.yaml#L708-L759

https://github.com/pahud/ecs-cfn-refarch/blob/09433d07b62cc711cce7cbe91487eecf123b0c7a/cloudformation/service.yaml#L820-L967

sibs786 commented 5 years ago

Hi Please confirm

This is my Auto Scaling Group

`  ECSAutoScalingGroup:
    DependsOn: ECSCluster
    Type: AWS::AutoScaling::AutoScalingGroup
    Properties:
      MixedInstancesPolicy:
        InstancesDistribution:
          OnDemandAllocationStrategy: prioritized
          OnDemandBaseCapacity: 0
          OnDemandPercentageAboveBaseCapacity: 50
          SpotAllocationStrategy: lowest-price
        LaunchTemplate:
          LaunchTemplateSpecification: 
            LaunchTemplateId: !Ref ECSLaunchConfiguration     
      VPCZoneIdentifier: !Ref Subnets
      MinSize: 2
      MaxSize: 5
      DesiredCapacity: 2
      Tags:
        - Key: Name
          Value: !Sub ${EnvironmentName} ECS host
          PropagateAtLaunch: true
    CreationPolicy:
      ResourceSignal:
        Timeout: PT15M
    UpdatePolicy:
      AutoScalingRollingUpdate:
        MinInstancesInService: 1
        MaxBatchSize: 1
        PauseTime: PT15M
        SuspendProcesses:
          - HealthCheck
          - ReplaceUnhealthy
          - AZRebalance
          - AlarmNotification
          - ScheduledActions
        WaitOnResourceSignals: true`

This is my launch template

`  ECSLaunchConfiguration:
    Type: AWS::AutoScaling::LaunchTemplate
    Properties:
      ImageId: !Ref ECSAMI
      InstanceType: !Ref InstanceType
      SecurityGroups:
        - !Ref SecurityGroup
      IamInstanceProfile: !Ref ECSInstanceProfile
      UserData:
        "Fn::Base64": !Sub |
          #!/bin/bash
          yum install -y https://s3.amazonaws.com/ec2-downloads-windows/SSMAgent/latest/linux_amd64/amazon-ssm-agent.rpm
          yum install -y https://s3.amazonaws.com/amazoncloudwatch-agent/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm
          yum install -y aws-cfn-bootstrap hibagent 
          /opt/aws/bin/cfn-init -v --region ${AWS::Region} --stack ${AWS::StackName} --resource ECSLaunchConfiguration
          /opt/aws/bin/cfn-signal -e $? --region ${AWS::Region} --stack ${AWS::StackName} --resource ECSAutoScalingGroup
          /usr/bin/enable-ec2-spot-hibernation

    Metadata:
      AWS::CloudFormation::Init:
        config:
          packages:
            yum:
              collectd: []

          commands:
            01_add_instance_to_cluster:
              command: !Sub echo ECS_CLUSTER=${ECSCluster} >> /etc/ecs/ecs.config
            02_enable_cloudwatch_agent:
              command: !Sub /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c ssm:${ECSCloudWatchParameter} -s
          files:
            /etc/cfn/cfn-hup.conf:
              mode: 000400
              owner: root
              group: root
              content: !Sub |
                [main]
                stack=${AWS::StackId}
                region=${AWS::Region}

            /etc/cfn/hooks.d/cfn-auto-reloader.conf:
              content: !Sub |
                [cfn-auto-reloader-hook]
                triggers=post.update
                path=Resources.ECSLaunchConfiguration.Metadata.AWS::CloudFormation::Init
                action=/opt/aws/bin/cfn-init -v --region ${AWS::Region} --stack ${AWS::StackName} --resource ECSLaunchConfiguration

          services:
            sysvinit:
              cfn-hup:
                enabled: true
                ensureRunning: true
                files:
                  - /etc/cfn/cfn-hup.conf
                  - /etc/cfn/hooks.d/cfn-auto-reloader.conf`
sibs786 commented 5 years ago

I just tested this, it gave me Template format error: Unrecognized resource types: [AWS::AutoScaling::LaunchTemplate]

This error i know i have made mistake it should be AWS::EC2:LaunchTemplate

sibs786 commented 5 years ago

What is the use of this


            if [ "$ilc" == "spot" ]; then 
               echo ECS_INSTANCE_ATTRIBUTES='{"instance-purchase-option":"spot"}' >> /etc/ecs/ecs.config 
             else 
               echo ECS_INSTANCE_ATTRIBUTES='{"instance-purchase-option":"ondemand"}' >> /etc/ecs/ecs.config 
             fi 
sibs786 commented 5 years ago

[Encountered unsupported properties in {/}: [SecurityGroups, UserData, ImageId, IamInstanceProfile, InstanceType]]

Please suggest what is important? i am going to try userdata you shared

sibs786 commented 5 years ago

Solved Problem