Simplify setup and teardown of staging infrastructure

[dennyabrain]

Scope of Work

script to start and stop a staging environment take a backup and restore aws rds postgres put secrets somewhere and checkin all other files kubectl comands to bring up and down deployment

[dennyabrain]

Lets start with 1: Write a script or command that can create an S3 bucket and Delete it 2: Create an IAM account programatically with suitable policy to read/write to this bucket

[dennyabrain]

Blocked at Move personal account to organization. AWS recommendation is to not use root account for everything else.

Next Steps :

• Switch to Organization IAM center
• Create new admin who can do the restricted operations

[duggalsu]

Managed to delete personal IAM Identity Center and recreate with AWS Organizations. Successfully added admin user with AdministratorAccess permissions. Refer: https://docs.aws.amazon.com/singlesignon/latest/userguide/quick-start-default-idc.html

[duggalsu]

Setup update

1. Enable IAM Identity Center with AWS Organizations for SSO. Any roles/users created are assigned temporary keys for security. (No need to worry about manually rotating keys on compromise).
2. Create an admin user for all tasks. Do not use the root user
3. Setup AWS CLI to allow access to user with SSO - since the access keys are temporary for security See instructions here - https://docs.aws.amazon.com/cli/latest/userguide/sso-configure-profile-token.html Basically need to run $ aws configure sso
4. Use AWS CLI to create S3 buckets similar to creating a bucket via console with all non-optional settings Refer CLI commands - https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3api/index.html#cli-aws-s3api

   # create bucket
   $ aws s3api create-bucket \
   --region us-east-1 \
   --bucket my-bucket \
   --object-ownership BucketOwnerEnforced \
   --no-object-lock-enabled-for-bucket

# list all buckets
$ aws s3 ls

# get public access block - this defaults to enabled for all 4 settings - so no need to manually enable
$ aws s3api get-public-access-block \
    --bucket my-bucket

• https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3api/put-bucket-versioning.html

  If the versioning state has never been set on a bucket, it has no versioning state; a GetBucketVersioning request does not return a versioning state value.

# get bucket versioning
$ aws s3api get-bucket-versioning \
    --bucket my-bucket

# put bucket versioning
$ aws s3api put-bucket-versioning \
    --bucket my-bucket \
    --versioning-configuration Status=Suspended

5. Rotate all permanent keys for security - without taking down services Refer
   • https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_revoke-sessions.html
   • https://docs.aws.amazon.com/IAM/latest/UserGuide/security-creds.html
   • https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html#rotate-credentials
   • https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html

[duggalsu]

Login to expired session (due to timeout)

1. Sign into account
2. Grant access to CLI by following instructions when executing

   $ aws sso login --no-browser

[duggalsu]

Task: Create an IAM account programatically with suitable policy to read/write to this bucket

https://aws.amazon.com/blogs/security/use-iam-identity-center-apis-to-audit-and-manage-application-assignments/

Before creating user - enable automatic sending of verification email via IAM Identity Center console

• https://docs.aws.amazon.com/singlesignon/latest/userguide/userswithoutpwd.html

# Step 1: Get your Identity Center instance information
$ aws sso-admin list-instances

# Step 2: Create user and group in your Identity Store
$ aws identitystore create-user --identity-store-id "d-**********" --user-name "MyUser" --emails Value="MyUser@example.com",Type="Work",Primary=true —display-name "My User" —name FamilyName="User",GivenName="My"

# Create a group in your Identity Store:
$ aws identitystore create-group --identity-store-id d-********** --display-name test-s3-app

# Run the following command to add the user to the group:
$ aws identitystore create-group-membership --identity-store-id d-********** --group-id ********-****-****-****-************ --member-id UserId=********-****-****-****-************

Create S3 policy https://docs.aws.amazon.com/AmazonS3/latest/userguide/example-policies-s3.html#iam-policy-ex2

Attaching the following policy to the group grants everybody in the group access to the following folder in Amazon S3: DOC-EXAMPLE-BUCKET1/share/marketing. Group members are allowed to access only the specific Amazon S3 permissions shown in the policy and only for objects in the specified folder. 

{
   "Version":"2012-10-17",
   "Statement":[
      {
         "Effect":"Allow",
         "Action":[
            "s3:PutObject",
            "s3:GetObject",
            "s3:GetObjectVersion",
            "s3:DeleteObject",
            "s3:DeleteObjectVersion"
         ],
         "Resource":"arn:aws:s3:::DOC-EXAMPLE-BUCKET1/share/marketing/*"
      }
   ]
}

TODO: https://docs.aws.amazon.com/singlesignon/latest/userguide/permissionsetsconcept.html

IAM Identity Center assigns access to a user or group in one or more AWS accounts with permission sets. 

You can add AWS managed policies, customer managed policies, inline policies, and AWS managed policies for job functions to your permission sets. You can also assign an AWS managed policy or a customer managed policy as a permissions boundary.

• https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sso-admin/create-permission-set.html
• https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sso-admin/create-account-assignment.html
• https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sso-admin/put-inline-policy-to-permission-set.html

Edit: Updated with instruction link for enabling automatic sending of verification email when creating user

[duggalsu]

Create permission set and assign to group

1. Create permission set https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sso-admin/create-permission-set.html

   $ aws sso-admin create-permission-set \
   --instance-arn <value> \
   --name <value>

2. Validate policy https://awscli.amazonaws.com/v2/documentation/api/latest/reference/accessanalyzer/validate-policy.html

   $ aws accessanalyzer validate-policy \
   --policy-document <value> \
   --policy-type <value>

3. Put inline policy to permission set https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sso-admin/put-inline-policy-to-permission-set.html

   $ aws sso-admin put-inline-policy-to-permission-set \
   --inline-policy <value> \
   --instance-arn <value> \
   --permission-set-arn <value>

4. List permission set provisioning status for verification https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sso-admin/list-permission-set-provisioning-status.html

   $ aws sso-admin list-permission-set-provisioning-status \
   --instance-arn <value>

5. Get group id for next command https://awscli.amazonaws.com/v2/documentation/api/latest/reference/identitystore/list-groups.html

   $ aws identitystore list-groups \
   --identity-store-id <value>

6. Assign permission set to group https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sso-admin/create-account-assignment.html

   $ aws sso-admin create-account-assignment \
   --instance-arn <value> \
   --permission-set-arn <value> \
   --principal-id <value> \
   --principal-type <value> \
   --target-id <value> \
   --target-type <value>

7. List account assignment creation status for verification https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sso-admin/list-account-assignment-creation-status.html

   $ aws sso-admin list-account-assignment-creation-status \
   --instance-arn <value>

8. Modify existing policy (optional)

   1. Re-invoke put inline policy as above
   2. Provision permission set to update policy for provisioned accounts https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sso-admin/provision-permission-set.html

      $ aws sso-admin provision-permission-set \
      --instance-arn <value> \
      --permission-set-arn <value> \
      --target-id <value> \
      --target-type <value>

   3. Verify provisioning status https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sso-admin/list-permission-set-provisioning-status.html

      $ aws sso-admin list-permission-set-provisioning-status \
      --instance-arn <value>

[duggalsu]

Now you can verify the user via email and set a password and add the user for AWS CLI access

[duggalsu]

In Progress - IAM Identity Center (SSO) and Github Actions to upload to S3

• https://dev.to/aws-builders/aws-sso-github-setup-35ld
• https://medium.com/@alissonpdc/github-actions-how-to-authenticate-on-aws-cloud-14185f811bf0
• https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services
• https://github.com/aws-actions/configure-aws-credentials

[duggalsu]

Update:

• https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers.html

  As a security best practice, we recommend you manage human users in IAM Identity Center with an external SAML identity provider instead of using SAML federation in IAM. For information about specific situations where an IAM user is required, see When to create an IAM user (instead of a role).

• IAM Identity Center - should be used to manage all humans users either using IAM Identity Center (within AWS org) OR external SAML (Security Assertion Markup Language) identity provider

• IAM (roles - NOT users) - should be used for machine-to-machine IdP using OIDC eg github actions

Steps to follow to configure Github Actions access to AWS

• https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services

Official AWS credentials github action - https://github.com/aws-actions/configure-aws-credentials Create IAM OIDC identity - https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_oidc.html

• https://awscli.amazonaws.com/v2/documentation/api/latest/reference/iam/list-open-id-connect-providers.html

  $ aws iam list-open-id-connect-providers

  $ aws iam create-open-id-connect-provider \
  --url <value> \
  --client-id-list <value> \
  --tags Key=string,Value=string

Create roles for OIDC - https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-idp_oidc.html

• https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies.html#policies_resource-based

  Resource-based policies are JSON policy documents that you attach to a resource such as an Amazon S3 bucket. These policies grant the specified principal permission to perform specific actions on that resource and defines under what conditions this applies. Resource-based policies are inline policies. There are no managed resource-based policies.

Example steps - https://www.eliasbrange.dev/posts/secure-aws-deploys-from-github-actions-with-oidc/

TODO

1. Use Github environments - https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment

   Environments, environment secrets, and deployment protection rules are available in public repositories for all current GitHub plans. They are not available on legacy plans, such as Bronze, Silver, or Gold. For access to environments, environment secrets, and deployment branches in private or internal repositories, you must use GitHub Pro, GitHub Team, or GitHub Enterprise. If you are on a GitHub Free, GitHub Pro, or GitHub Team plan, other deployment protection rules, such as a wait timer or required reviewers, are only available for public repositories.

2. Configure environment in role trust policy - https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services#configuring-the-role-and-trust-policy

   
   If you use a workflow with an environment, the sub field must reference the environment name: repo:OWNER/REPOSITORY:environment:NAME. For more information, see "About security hardening with OpenID Connect."

Note: When environments are used in workflows or in OIDC policies, we recommend adding protection rules to the environment for additional security. For example, you can configure deployment rules on an environment to restrict which branches and tags can deploy to the environment or access environment secrets. For more information, see "Using environments for deployment."

3. Create IAM roles - https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-idp.html
Create role - https://awscli.amazonaws.com/v2/documentation/api/latest/reference/iam/create-role.html

$ aws iam create-role \ --role-name \ --assume-role-policy-document

Sample role trust policy document - https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-idp_oidc.html#idp_oidc_Create_GitHub

{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Federated": "arn:aws:iam::012345678910:oidc-provider/token.actions.githubusercontent.com" }, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": { "StringEquals": { "token.actions.githubusercontent.com:aud": "sts.amazonaws.com", "token.actions.githubusercontent.com:sub": "repo:GitHubOrg/GitHubRepo:ref:refs/heads/GitHubBranch" } } } ] }

Create inline permission policy - https://awscli.amazonaws.com/v2/documentation/api/latest/reference/iam/put-role-policy.html

$ aws iam put-role-policy \ --role-name \ --policy-name \ --policy-document

Sample role permissions policy - https://docs.aws.amazon.com/AmazonS3/latest/userguide/example-policies-s3.html#iam-policy-ex2

{ "Version":"2012-10-17", "Statement":[ { "Effect":"Allow", "Action":[ "s3:PutObject", "s3:GetObject", "s3:GetObjectVersion", "s3:DeleteObject", "s3:DeleteObjectVersion" ], "Resource":"arn:aws:s3:::DOC-EXAMPLE-BUCKET1/share/marketing/*" } ] }

[duggalsu]

Successfully setup github workflow to push objects to AWS S3. Refer sample workflow here

• https://github.com/duggalsu/aws-test/blob/main/.github/workflows/aws-s3-deploy-prod.yml

[duggalsu]

Validating the S3 bucket role policy

$ aws accessanalyzer validate-policy \
    --policy-document <value> \
    --policy-type RESOURCE_POLICY

[dennyabrain]

Denny's notes from Aurora's call

Create accounts/identities which have temporary credentials that get revoked every hour (default)

AWS has 3 products for identity management

1. IAM any user gets long lived credentials. never add or share publically never create users here (deprecated)
2. IAM Identity Center (called as Identity Provider, IdP) previously called Single Sign On. any user you create have short lived credentials. and they get rotated. standard practice is to create users from IAM identity center. if you want a subset of the users to have certain permissions, you make policies, add them to a group. add user to group.
   This is how you create human users.
3. Machine to machine interaction doesn't require a user. create roles, which have an identity and not associated with any user. role is similar to group. From a third party service. the third party service will act as an IdP. they will create an identity and then generally through a protocol like OIDC, you will be able to add this external identity as part of aws iam role. IdP doesnt allow creating roles.
4. AWS Cognito. IAM full fledged access management.

Convention create groups with a name according to role of the group.

Root user HAS to create an admin account or restricted user via IAM iDentity center. since these credentials are short lived

1. login through web browser
2. login through aws cli
   1. open in browser, authorize access.

users created via IdP, when the admin user logs in they see their access key.

aws CLI setup https://docs.aws.amazon.com/cli/latest/userguide/sso-configure-profile-token.html once setup, key rotation happens automatically. ! AWS CLI version 2

two types of policies trust policy defines who the user who gets access to this group or role attached when creating a role permissions policy what services and applications a user gets access to

Github Environments generally used for deployment sets check before deployment can progress within this environment you can add secrets only if the checks pass, secret access is blocked.

when using environment, url in the github policy will be different.

there is a cli to validate policies.

Generate the github OIDC token and added it to IAM role

Process

1. in aws create an OIDC provider linked to github
2. you create a role, and attach a trust policy, in the policy is details of the github OIDC provider and the repo details.

in create role step assume-role-policy-document -> is the trust policy put-role-policy-> permissions policy

inline policy vs managed policy

AWS_ROLE_TO_ASSUME is the role ARN

[duggalsu]

Note: Permissions

Approach 1: Role-Based Access Control (RBAC)

• https://docs.aws.amazon.com/IAM/latest/UserGuide/introduction_attribute-based-access-control.html#introduction_attribute-based-access-control_compare-rbac

  
  The traditional authorization model used in IAM is called role-based access control (RBAC). RBAC defines permissions based on a person's job function, known outside of AWS as a role. Within AWS a role usually refers to an IAM role, which is an identity in IAM that you can assume. IAM does include managed policies for job functions that align permissions to a job function in an RBAC model.

In IAM, you implement RBAC by creating different policies for different job functions. You then attach the policies to identities (IAM users, groups of users, or IAM roles). As a best practice, you grant the minimum permissions necessary for the job function. This is known as granting least privilege. Do this by listing the specific resources that the job function can access. The disadvantage to using the traditional RBAC model is that when employees add new resources, you must update policies to allow access to those resources.

Approach 2: Attribute-Based Access Control (ABAC)

- https://docs.aws.amazon.com/IAM/latest/UserGuide/tutorial_attribute-based-access-control.html

Attribute-based access control (ABAC) is an authorization strategy that defines permissions based on attributes. In AWS, these attributes are called tags.

- https://docs.aws.amazon.com/IAM/latest/UserGuide/introduction_attribute-based-access-control.html#introduction_attribute-based-access-control_compare-rbac

ABAC provides the following advantages over the traditional RBAC model:

ABAC permissions scale with innovation. It's no longer necessary for an administrator to update existing policies to allow access to new resources. For example, assume that you designed your ABAC strategy with the access-project tag. A developer uses the role with the access-project = Heart tag. When people on the Heart project need additional Amazon EC2 resources, the developer can create new Amazon EC2 instances with the access-project = Heart tag. Then anyone on the Heart project can start and stop those instances because their tag values match.

ABAC requires fewer policies. Because you don't have to create different policies for different job functions, you create fewer policies. Those policies are easier to manage.

Using ABAC, teams can change and grow quickly. This is because permissions for new resources are automatically granted based on attributes. For example, if your company already supports the Heart and Star projects using ABAC, it's easy to add a new Lightning project. An IAM administrator creates a new role with the access-project = Lightning tag. It's not necessary to change the policy to support a new project. Anyone that has permissions to assume the role can create and view instances tagged with access-project = Lightning. Additionally, a team member might move from the Heart project to the Lightning project. The IAM administrator assigns the user to a different IAM role. It's not necessary to change the permissions policies.

Granular permissions are possible using ABAC. When you create policies, it's a best practice to grant least privilege. Using traditional RBAC, you must write a policy that allows access to only specific resources. However, when you use ABAC, you can allow actions on all resources, but only if the resource tag matches the principal's tag.

Use employee attributes from your corporate directory with ABAC. You can configure your SAML or OIDC provider to pass session tags to AWS. When your employees federate into AWS, their attributes are applied to their resulting principal in AWS. You can then use ABAC to allow or deny permissions based on those attributes.

ABAC in IAM Identity Center
- https://docs.aws.amazon.com/singlesignon/latest/userguide/abac.html

The following are additional benefits of using ABAC in IAM Identity Center.

ABAC requires fewer permission sets – Because you don't have to create different policies for different job functions, you create fewer permission sets. This reduces your permissions management complexity.

Using ABAC, teams can change and grow quickly – Permissions for new resources are automatically granted based on attributes when resources are appropriately tagged upon creation.

Use employee attributes from your corporate directory with ABAC – You can use existing employee attributes from any identity source configured in IAM Identity Center to make access control decisions in AWS.

Track who is accessing resources – Security administrators can easily determine the identity of a session by reviewing the user attributes in AWS CloudTrail to track user activity in AWS.

TODO
- https://docs.aws.amazon.com/singlesignon/latest/userguide/abac-checklist.html
- https://docs.aws.amazon.com/singlesignon/latest/userguide/attributesforaccesscontrol.html
- https://docs.aws.amazon.com/singlesignon/latest/APIReference/API_CreateInstanceAccessControlAttributeConfiguration.html

[dennyabrain]

@duggalsu ok so this makes sense. Lets finalize a convention around Attributes to create some policies. I can think of two attributes that are important for access control - Project Name and Environment. For eg, Aatman can create/edit/delete resources under Uli and Staging environment. or Denny can create/edit/delete resources under Viral Spiral and Staging and Production environment etc. Let me know if you can think of many other attributes.

This also means that in all of out automations and scripts we'll have to make sure we add tags to resources.