ec2:TerminateInstances already included in execution role policy

tadeasbronis commented 3 years ago

I tried to use in our company, but with my restricted access I ended up with an error: You do not have permissions to make the TerminateInstances API call. Please try again with the necessary permissions.

I checked your code and found out, that the error is thrown because You are trying to add ec2:TerminateInstances in CreateHandler.java again. But ec2:TerminateInstances permissions are already added in execution role policy. Can You please delete that of the code? Thanks.

maslick commented 3 years ago

Same issue here...

maslick commented 3 years ago

He's not adding the policy, but rather simulates if the requester principal (IAM user or role) has enough rights to ec2:TerminateInstances. I have no idea why this check is needed... Apparently he wants to check if it will be possible to teardown the CommandRunner EC2 instance before spinning it up. It's strange indeed, because the execution role created after registering the CommandRunner extension already has this policy attached:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "cloudformation:CreateStack",
                "cloudformation:DeleteStack",
                "cloudformation:DescribeStacks",
                "ec2:AuthorizeSecurityGroupEgress",
                "ec2:AuthorizeSecurityGroupIngress",
                "ec2:CreateSecurityGroup",
                "ec2:CreateTags",
                "ec2:DeleteSecurityGroup",
                "ec2:DescribeInstances",
                "ec2:DescribeSecurityGroups",
                "ec2:DescribeSubnets",
                "ec2:DescribeVpcs",
                "ec2:RevokeSecurityGroupEgress",
                "ec2:RevokeSecurityGroupIngress",
                "ec2:RunInstances",
                "ec2:TerminateInstances",
                "iam:PassRole",
                "iam:SimulatePrincipalPolicy",
                "kms:Decrypt",
                "kms:Encrypt",
                "logs:CreateLogStream",
                "logs:DescribeLogGroups",
                "logs:PutMetricData",
                "ssm:DeleteParameter",
                "ssm:GetParameter",
                "ssm:PutParameter",
                "sts:GetCallerIdentity"
            ],
            "Resource": "*",
            "Effect": "Allow"
        }
    ]
}

maslick commented 2 years ago

This PR #30 solves the issue

jlongo62 commented 2 years ago

So if this thing does not work how do we get maslick's changes integrated into a release ?

maslick commented 2 years ago

@jlongo62 i don't think this repo is maintained...

jlongo62 commented 2 years ago

are you using this ? do you have it working ?

maslick commented 2 years ago

@jlongo62 yes, we are using this in production

jlongo62 commented 2 years ago

what is the workaround ?

maslick commented 2 years ago

@jlongo62 see #30

jlongo62 commented 2 years ago

how dows #30 help me ? there is no release that incorporates it ...

maslick commented 2 years ago

@jlongo62 so what? use the feature branch and follow the instructions:

$ ./scripts/build.sh

jlongo62 commented 2 years ago

I can do the git and get the pull request. But do you have a script to install the dependencies on AWS cloudshell to be able to process ./scripts/build.sh

maslick commented 2 years ago

@jlongo62 you need:

Java 1.8+
Maven 3+
cfn cli
AWS CLI v2

# Install cfn cli
pip install cloudformation-cli cloudformation-cli-java-plugin cloudformation-cli-go-plugin cloudformation-cli-python-plugin cloudformation-cli-typescript-plugin

# Clone maslick's fork of CommandRunner, checkout fix-terminate-instances branch
git clone git@github.com:maslick/aws-cloudformation-resource-providers-awsutilities-commandrunner.git
git checkout fix-terminate-instances

# Login to your AWS account, generate necessary "helper" classes, build and deploy CRD to Cloudformation
aws configure
scripts/build.sh

jlongo62 commented 2 years ago

Any chance of you publishing a release for this in your repository ?

jlongo62 commented 2 years ago

I spent an ENTIRE day getting this thing compiled and deployed and tested.

Gotchas: Make sure you have a default VPC. Make sure there is just one subnet in the availability zone that can create a t3.micro I tested the command in a Linux window I have a simple resourrce that works

IopsCalculator:
  Type: AWSUtility::CloudFormation::CommandRunner
  Properties:
    Role: InfrastructureManagement
    Command:
      Fn::Sub: 'expr 100 \* 20 > /command-output.txt'

jlongo62 commented 2 years ago

Now I cannot get the aws commands to work. I have a simple resource straight from the docs:

    GetransitGatewayRouteTableId:
      Type: AWSUtility::CloudFormation::CommandRunner
      DependsOn: [VPNConnection]
      Properties: 
        Role: InfrastructureManagement
        LogGroup: "awsutility-cloudformation-commandrunner-logs"
        Command: 'aws s3 ls | sed -n 1p | cut -d " " -f3 > /command-output.txt'

It returns the error:

Resource handler returned message: "1 validation error detected: Value at 'value' failed to satisfy constraint: Member must have length greater than or equal to 1. (Service: AWSSimpleSystemsManagement; Status Code: 400; Error Code: ValidationException; Request ID: 915da8da-06f2-49f0-80da-0b6d7140a73b; Proxy: null)" (RequestToken: 9410d74b-a60e-f604-d7d5-6344199b7de3, HandlerErrorCode: GeneralServiceException)

maslick commented 2 years ago

@jlongo62 you should also consider this #32

Can you use this template?

AWSTemplateFormatVersion: 2010-09-09
Description: AWS CLI version

Resources:
  LogGroup:
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupName: !Sub "/command-runner/${AWS::StackName}/"
      RetentionInDays: 14

  Role:
    Type: AWS::IAM::Role
    Properties:
      Description: "Role assumed by Command Runnner"
      MaxSessionDuration: 14400
      Path: "/"
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: ec2.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy

  InstanceProfile:
    Type: AWS::IAM::InstanceProfile
    Properties:
      Path: /
      Roles:
        - !Ref Role

  CommandRunner:
    Type: AWSUtility::CloudFormation::CommandRunner
    Properties:
      LogGroup: !Ref LogGroup
      Role: !Ref InstanceProfile
      SubnetId: YOUR_SUBNET_ID
      Command: |
        aws --version 2>&1 \
          | tail -1 \
          | head -n 1 \
          > /command-output.txt

Outputs:
  AwsCliVersion:
    Description: AWS CLI version
    Value: !GetAtt CommandRunner.Output

Check the logs from /command-runner/${AWS::StackName}/ log group

maslick commented 2 years ago

@jlongo62 you are getting this error, probably because your role doesn't have an associated instance profile. An Instance profile is created automatically when you create an EC2 instance from the console. CommandRunner is expecting the Instance profile, and not a plain Role in the Role property. This field name is counterintuitive and should be rather renamed to InstanceProfile. You are absolutely right that if you do not specify any subnet your account must have a default VPC and the region should support t3.micro instance type.

maslick commented 2 years ago

BTW we are using this branch (from #32) in production, since we don't have a default VPC, and we need to run CommandRunner in a private subnet.

I also explicitly set LogGroup (to be able to debug the output created by CommandRunner in CloudWatch), the Role (in fact it's an Instance Profile) should have the necessary policies to be able to write to CloudWatch and a SubnetId (we are running our services mostly in a private subnet of a non-default VPC).

HTH

jlongo62 commented 2 years ago

The first problem is the code at https://github.com/aws-cloudformation/aws-cloudformation-resource-providers-awsutilities-commandrunner.git does not work... the repository is abandoned.

maslick's repository listed in the accompanying shell does work, if you pull the correct branch (also in this shell script). if you pull the correct branch. You will also need the correct role, also in this post.

Shell script to install commandrunner using cloudshell:

############################################################################
# This script Configures Cloudshell and Builds and Deploys CommandRunner
# This script should work on a Linux Box
# Cloudshell resets when you change Roles
# Cloudshell invokes AWS Client using the current Portal Role.
# You must be in the correct Role before attempting to deploy.
# Use the IAMManagement role depoyed by IAMManagement.yaml
# if you do not specify a Subnet you must have a Default VPC
# The subnet must be in an AZ that can support t3.micro
#############################################################################

#install java
sudo yum install java-1.8.0-openjdk-devel.x86_64 -y
java -version

#Following are the set of commands need to be executed sequentially to install maven.
sudo wget http://repos.fedorapeople.org/repos/dchen/apache-maven/epel-apache-maven.repo -O /etc/yum.repos.d/epel-apache-maven.repo
sudo sed -i s/\$releasever/6/g /etc/yum.repos.d/epel-apache-maven.repo
sudo yum install -y apache-maven
mvn –v

#Install Git
# --> GIT is already present in cloudshell
#sudo yum install git
#git version

# Install Python(LINUX box)
# https://tecadmin.net/install-python-3-8-amazon-linux/#:~:text=Installing%20Python%20on%20Amazon%20Linux%201%20Step%201,Environment%20on%20Ubuntu%2018.04%20%26%2016.04.%20See%20More.
cd /
sudo yum install gcc openssl-devel bzip2-devel libffi-devel  zlib-devel -y
cd /opt
sudo wget https://www.python.org/ftp/python/3.8.12/Python-3.8.12.tgz
sudo tar xzf Python-3.8.12.tgz
cd Python-3.8.12
sudo ./configure --enable-optimizations 
sudo make altinstall
sudo rm -f /opt/Python-3.8.12.tgz
python3.8 --version
python3 --version #(should be 3.8)

# Make Python3.8 default
# https://tech.serhatteker.com/post/2019-12/upgrade-python38-on-ubuntu/
whereis python3.8.12
sudo update-alternatives --install /usr/bin/python3 python3 /usr/local/bin/python3.8 1
sudo update-alternatives --set python3 /usr/local/bin/python3.8
python3 --version #(should be 3.8)

#Install cfn
pip3 install cloudformation-cli cloudformation-cli-java-plugin cloudformation-cli-go-plugin cloudformation-cli-python-plugin cloudformation-cli-typescript-plugin

#Install AWS CLI 2
# --> AWS CLI is already present in cloudshell
#curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
#unzip awscliv2.zip
#sudo yum install awscli -y
#aws --version
# AWAS CLI is already using current role in cloudshell
#aws configure

#get the source
#maslick has the working version.
#the git reffered to in AWS docs is unmaintained and does not work.
cd ~
git clone https://github.com/maslick/aws-cloudformation-resource-providers-awsutilities-commandrunner.git
cd ~/aws-cloudformation-resource-providers-awsutilities-commandrunner
git checkout fix-null-sg

#build it
alias python="/usr/bin/Python-3.8.12/python"
clear
cd ~/aws-cloudformation-resource-providers-awsutilities-commandrunner
./scripts/build.sh

The YAML to make an assumable role to run CloudShell undert is also included (I pulled it from another project, handy if you have to have and admin install)

AWSTemplateFormatVersion: "2010-09-09"
Metadata:
  Generator: "former2"
Description: "Provides permissions for managing Roles and Policies. Reqired by IAMTouchworks.yaml. Disable these roles by re-deploying and setting allowIAMAssumption=No"

Parameters:
  environment:
    Description: "Environment Name"
    Type: String
    Default: "Sandbox-1"

  environmentShort:
    Description: "Environment Name (Short)"
    Type: String
    Default:  "S1"

  allowIAMAssumption:
    Description: "Allow IAM Assumption Policy"
    Type: String
    Default:  "No"
    AllowedValues: 
      - "No"
      - "YES"
    ConstraintDescription: "Yes or No"

Conditions:
  IAMAssumptionCondition:  !Equals 
    - !Ref allowIAMAssumption
    - "YES"

Resources:

  IAMManagementPolicy:
    Type: "AWS::IAM::ManagedPolicy"
    Properties:
      ManagedPolicyName: IAMManagement
      Path: "/"
      PolicyDocument: |
        {
            "Version": "2012-10-17",
            "Statement": [
          {
                "Effect": "Allow",
                "Action": [
                "cloudformation:*",
                "cloudshell:*",
                "s3:*",
                "sns:*",
                "kms:*"
                ],
                "Resource": "*"
            },
          {
                "Effect": "Allow",
                "Action": [
                "ec2:DescribeAccountAttributes",
                    "sts:GetCallerIdentity",
                "access-analyzer:ListPolicyGenerations"
                ],
                "Resource": "*"
            },
          {
                "Effect": "Allow",
                "Action": [
                "iam:List*",
                    "iam:Get*",
                "iam:Update*",
                "iam:Delete*"
                ],
                "Resource": "*"
            },
            {
                "Effect": "Allow",
                "Action": [
                    "iam:AddRoleToInstanceProfile",
                    "iam:CreateInstanceProfile",
                    "iam:DeleteInstanceProfile",
                    "iam:GetInstanceProfile",
                    "iam:RemoveRoleFromInstanceProfile",
                    "iam:DeleteRolePolicy",
                    "iam:ListPolicies",
                    "iam:ListRoles",
                    "iam:ListUserPolicies",
                    "iam:CreateAccessKey",
                    "iam:UpdateAccessKey",
                    "iam:TagUser",
                    "iam:PutUserPolicy",
                    "iam:ListEntitiesForPolicy",
                    "iam:DeleteAccessKey",
                    "iam:ListAccessKeys",
                    "iam:ListAttachedUserPolicies",
                    "iam:ListSigningCertificates",
                    "iam:ListSSHPublicKeys",
                    "iam:ListServiceSpecificCredentials",
                    "iam:DeleteUserPolicy",
                    "iam:GenerateServiceLastAccessedDetails"
                ],
                "Resource": "*"
            },
            {
                "Effect": "Allow",
                "Action": [
                  "iam:AddUserToGroup",
                  "iam:AttachRolePolicy",
                  "iam:AttachUserPolicy",
                  "iam:CreateGroup",
                  "iam:CreateRole",
                  "iam:CreatePolicy",
                  "iam:CreatePolicyVersion",
                  "iam:CreateUser",
                  "iam:DeleteRole",
                  "iam:DeleteRolePolicy",
                  "iam:DetachRolePolicy",
                  "iam:DetachUserPolicy",
                  "iam:DeleteGroup",
                  "iam:DeleteGroupPolicy",
                  "iam:DeleteUser",
                  "iam:GetGroup",
                  "iam:GetRole",
                  "iam:GetRolePolicy",
                  "iam:GetPolicy",
                  "iam:GetUser",
                  "iam:DeletePolicy",
                  "iam:GetPolicyVersion",
                  "iam:ListAttachedRolePolicies",
                  "iam:ListInstanceProfilesForRole",
                  "iam:ListRolePolicies",
                  "iam:ListPolicyVersions",
                  "iam:ListGroups",
                  "iam:ListUsers",
                  "iam:DeletePolicyVersion",
                  "iam:PassRole",
                  "iam:PutRolePolicy",
                  "iam:PutGroupPolicy",
                  "iam:RemoveUserFromGroup",
                  "iam:TagRole"
                ],
                "Resource": "*"
            }]
        }

  IAMManagementRole:
    Type: "AWS::IAM::Role"
    Properties:
      Path: "/"
      RoleName: "IAMManagement"
      AssumeRolePolicyDocument:  
        Fn::If: 
          - IAMAssumptionCondition
          - Fn::Sub: |
              {
                "Version": "2012-10-17",
                "Statement": [
                  {
                    "Effect": "Allow",
                    "Principal": {
                      "AWS": ["arn:aws:iam::${AWS::AccountId}:role/CSS-Customer-Admin"]
                  },
                  "Action": "sts:AssumeRole"
                }
                ]
              }
          - Fn::Sub: |
              {
                "Version": "2012-10-17",
                "Statement": [
                  {
                    "Effect": "Deny",
                    "Principal": {
                      "AWS": [
                        "*"
                      ]
                  },
                  "Action": "sts:AssumeRole"
                }
                ]
              }

      MaxSessionDuration: 3600
      ManagedPolicyArns:
        - !Ref IAMManagementPolicy
      Tags:
        - Key: "Environment"
          Value: 
            Ref: environment
        - Key: "Environment-Short"
          Value: 
            Ref: environmentShort
        - Key: "Purpose"
          Value: "Role"
        - Key: "Application"
          Value: "Infrastructure"
        - Key: "Description"
          Value: "Allows IAM Role-Policy Management"

A non-trivial, but simple test CF to verify commandRunner can use the AWS CLI.

#!!! THE RELEASE DOES NOT WORK !!!!
#To actually get this to work, you have to follow these instructions to build the tool youreself.
#https://github.com/aws-cloudformation/aws-cloudformation-resource-providers-awsutilities-commandrunner/issues/24
#https://github.com/maslick/aws-cloudformation-resource-providers-awsutilities-commandrunner/tree/fix-null-sg

#CommandRunner Documentation
#https://aws.amazon.com/premiumsupport/knowledge-center/cloudformation-commandrunner-stack/
#https://github.com/aws-cloudformation/aws-cloudformation-resource-providers-awsutilities-commandrunner#prerequisites

# This script Assumes Default VPC Exists

AWSTemplateFormatVersion: 2010-09-09
Description: AWS CLI version

Resources:
  LogGroup:
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupName: !Sub "/command-runner/${AWS::StackName}/"
      RetentionInDays: 14

  Role:
    Type: AWS::IAM::Role
    Properties:
      Description: "Role assumed by Command Runnner"
      MaxSessionDuration: 14400
      Path: "/"
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: ec2.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy
        - arn:aws:iam::aws:policy/AmazonS3FullAccess

  InstanceProfile:
    Type: AWS::IAM::InstanceProfile
    Properties:
      Path: /
      Roles:
        - !Ref Role
  CommandRunner:
    Type: AWSUtility::CloudFormation::CommandRunner
    Properties:
      LogGroup: !Ref LogGroup
      Role: !Ref InstanceProfile
      #SubnetId: YOUR_SUBNET_ID
      Command: |
        aws s3 ls | sed -n 1p | cut -d " " -f3 \
          > /command-output.txt

      # aws --version 2>&1 \
      #   | tail -1 \
      #   | head -n 1 \
      #   > /command-output.txt

      # aws s3 ls | sed -n 1p | cut -d " " -f3  > /command-output.txt

Outputs:
  CommandRunnerOutput:
    Description: CommandRunnerOutput
    Value: !GetAtt CommandRunner.Output

aws-cloudformation / aws-cloudformation-resource-providers-awsutilities-commandrunner

ec2:TerminateInstances already included in execution role policy #24