Authenticate from GitHub via OIDC

[douglasnaphas]

https://github.com/douglasnaphas/just-a-table/issues/1, https://github.com/VerticalRelevance/aws-github-oidc, and friends.

[douglasnaphas]

TODO

• [x] Check out the trust policy on the CDK cfn-exec-role shown in this comment. CloudFormation service principal.
• [ ] Something with automating/facilitating the getting of the necessary prod actions. Maybe make an Actions step that asserts that the actions that the dev role has taken are included in the prod role's policy. Nota bene: There are complications with this. I think the action names I get out of CloudTrail and out of get-service-last-accessed-details are not exactly the same as IAM actions. Also the prod role needs privileges to delete things as well as create an update them. So maybe make a tool to facilitate getting the required permissions.
• [ ] Use aws-github-oidc-role.

[douglasnaphas]

Scoped-down permissions for higher env roles

I've decided the test and prod roles should have least-privilege.

The recommended way to do this is by deploying with an admin role to dev, then checking CloudTrail (or maybe IAM Access Advisor, or aws iam generate-service-last-accessed-details if that's different, would be easier?) to see what perms are needed.

I had a problem where eventsource and eventname, and most other fields, came back blank for CloudTrail events where the role was arn:aws:iam::287213416300:role/github-actions@madliberation. See row 236 here:

This was the case even though eventSource and eventName were present in the underlying JSON files:

This turned out to be because the recommended query here had sessionContext.webIdFederationData as map<string,string>

while the events where the role was arn:aws:iam::287213416300:role/github-actions@madliberation had a non-string value for one of the map entries, as pictured above:

I removed webIdFederationData:map<string,string> from the CREATE EXTERNAL TABLE statement, and the following sequence of queries showed me the actions taken by my GitHub Actions role in dev.

CREATE EXTERNAL TABLE cloudtrail_logs_aws_controltower_logs_636074447003_us_east_1_org_part_acct4 (
    eventversion STRING,
    useridentity STRUCT < type: STRING,
    principalid: STRING,
    arn: STRING,
    accountid: STRING,
    invokedby: STRING,
    accesskeyid: STRING,
    userName: STRING,
    sessioncontext: STRUCT < attributes: STRUCT < mfaauthenticated: STRING,
    creationdate: STRING >,
    sessionissuer: STRUCT < type: STRING,
    principalId: STRING,
    arn: STRING,
    accountId: STRING,
    userName: STRING >,
    ec2RoleDelivery: string > >,
    eventtime STRING,
    eventsource STRING,
    eventname STRING,
    awsregion STRING,
    sourceipaddress STRING,
    useragent STRING,
    errorcode STRING,
    errormessage STRING,
    requestparameters STRING,
    responseelements STRING,
    additionaleventdata STRING,
    requestid STRING,
    eventid STRING,
    resources ARRAY < STRUCT < arn: STRING,
    accountid: STRING,
    type: STRING >>,
    eventtype STRING,
    apiversion STRING,
    readonly STRING,
    recipientaccountid STRING,
    serviceeventdetails STRING,
    sharedeventid STRING,
    vpcendpointid STRING,
    eventCategory STRING,
    tlsDetails struct < tlsVersion: string,
    cipherSuite: string,
    clientProvidedHostHeader: string >
) COMMENT 'CloudTrail table for aws-controltower-logs-636074447003-us-east-1 bucket'
PARTITIONED BY (
    account string,
    region string,
    year string,
    month string
)
ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe'
STORED AS INPUTFORMAT 'com.amazon.emr.cloudtrail.CloudTrailInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION 's3://aws-controltower-logs-636074447003-us-east-1/o-w99rhe3hsv/AWSLogs/o-w99rhe3hsv/'
TBLPROPERTIES ('classification' = 'cloudtrail');

ALTER TABLE `cloudtrail_logs_aws_controltower_logs_636074447003_us_east_1_org_part_acct4`
ADD PARTITION (
        account = '287213416300',
        region = 'us-east-1',
        year = '2024',
        month = '02'
    )
    LOCATION 's3://aws-controltower-logs-636074447003-us-east-1/o-w99rhe3hsv/AWSLogs/o-w99rhe3hsv/287213416300/CloudTrail/us-east-1/2024/02/'
    ;

SELECT COUNT(*),
    useridentity.sessionContext.sessionIssuer.arn AS principal,
    eventsource,
    eventname
FROM "default"."cloudtrail_logs_aws_controltower_logs_636074447003_us_east_1_org_part_acct4"
WHERE useridentity.sessionContext.sessionIssuer.arn = 'arn:aws:iam::287213416300:role/github-actions@madliberation'
GROUP BY useridentity.sessionContext.sessionIssuer.arn, eventsource, eventname
ORDER BY eventsource, eventname;

The results are

#   _col0   principal   eventsource eventname
1   1   arn:aws:iam::287213416300:role/github-actions@madliberation cloudformation.amazonaws.com    CreateChangeSet
2   1   arn:aws:iam::287213416300:role/github-actions@madliberation cloudformation.amazonaws.com    DeleteChangeSet
3   3   arn:aws:iam::287213416300:role/github-actions@madliberation cloudformation.amazonaws.com    DescribeChangeSet
4   7   arn:aws:iam::287213416300:role/github-actions@madliberation cloudformation.amazonaws.com    DescribeStackEvents
5   49  arn:aws:iam::287213416300:role/github-actions@madliberation cloudformation.amazonaws.com    DescribeStacks
6   1   arn:aws:iam::287213416300:role/github-actions@madliberation cloudformation.amazonaws.com    ExecuteChangeSet
7   5   arn:aws:iam::287213416300:role/github-actions@madliberation cloudformation.amazonaws.com    GetTemplate
8   3   arn:aws:iam::287213416300:role/github-actions@madliberation cognito-idp.amazonaws.com   AdminCreateUser
9   3   arn:aws:iam::287213416300:role/github-actions@madliberation cognito-idp.amazonaws.com   DescribeUserPool
10  1   arn:aws:iam::287213416300:role/github-actions@madliberation iam.amazonaws.com   GetRole
11  1   arn:aws:iam::287213416300:role/github-actions@madliberation iam.amazonaws.com   ListAttachedRolePolicies
12  1   arn:aws:iam::287213416300:role/github-actions@madliberation iam.amazonaws.com   ListRolePolicies
13  1   arn:aws:iam::287213416300:role/github-actions@madliberation iam.amazonaws.com   PutRolePolicy
14  2   arn:aws:iam::287213416300:role/github-actions@madliberation s3.amazonaws.com    GetAccelerateConfiguration
15  2   arn:aws:iam::287213416300:role/github-actions@madliberation s3.amazonaws.com    GetBucketAnalyticsConfiguration
16  2   arn:aws:iam::287213416300:role/github-actions@madliberation s3.amazonaws.com    GetBucketCors
17  2   arn:aws:iam::287213416300:role/github-actions@madliberation s3.amazonaws.com    GetBucketEncryption
18  2   arn:aws:iam::287213416300:role/github-actions@madliberation s3.amazonaws.com    GetBucketIntelligentTieringConfiguration
19  2   arn:aws:iam::287213416300:role/github-actions@madliberation s3.amazonaws.com    GetBucketInventoryConfiguration
20  2   arn:aws:iam::287213416300:role/github-actions@madliberation s3.amazonaws.com    GetBucketLifecycle
21  2   arn:aws:iam::287213416300:role/github-actions@madliberation s3.amazonaws.com    GetBucketLogging
22  2   arn:aws:iam::287213416300:role/github-actions@madliberation s3.amazonaws.com    GetBucketMetricsConfiguration
23  2   arn:aws:iam::287213416300:role/github-actions@madliberation s3.amazonaws.com    GetBucketNotification
24  2   arn:aws:iam::287213416300:role/github-actions@madliberation s3.amazonaws.com    GetBucketObjectLockConfiguration
25  2   arn:aws:iam::287213416300:role/github-actions@madliberation s3.amazonaws.com    GetBucketOwnershipControls
26  2   arn:aws:iam::287213416300:role/github-actions@madliberation s3.amazonaws.com    GetBucketPublicAccessBlock
27  2   arn:aws:iam::287213416300:role/github-actions@madliberation s3.amazonaws.com    GetBucketReplication
28  10  arn:aws:iam::287213416300:role/github-actions@madliberation s3.amazonaws.com    GetBucketTagging
29  2   arn:aws:iam::287213416300:role/github-actions@madliberation s3.amazonaws.com    GetBucketVersioning
30  2   arn:aws:iam::287213416300:role/github-actions@madliberation s3.amazonaws.com    GetBucketWebsite
31  2   arn:aws:iam::287213416300:role/github-actions@madliberation s3.amazonaws.com    PutBucketTagging
32  3   arn:aws:iam::287213416300:role/github-actions@madliberation sns.amazonaws.com   GetSMSSandboxAccountStatus
33  12  arn:aws:iam::287213416300:role/github-actions@madliberation ssm.amazonaws.com   GetParameters
34  1   arn:aws:iam::287213416300:role/github-actions@madliberation ssm.amazonaws.com   PutParameter
35  44  arn:aws:iam::287213416300:role/github-actions@madliberation sts.amazonaws.com   AssumeRole
36  60  arn:aws:iam::287213416300:role/github-actions@madliberation sts.amazonaws.com   GetCallerIdentity

I don't see things like DynamoDB CreateTable in there. I don't know whether the role I execute the change set with does that, or whether CloudFormation does that.

[douglasnaphas]

I provided the following feedback on this page.

The section "Creating a table for CloudTrail logs in Athena using manual partitioning" includes a line in the provided CREATE EXTERNAL TABLE statement:

webIdFederationData:map<string,string>

However, for my CloudTrail trail, a role assumed by a GitHub Actions OpenID Connect provider had the following:

"webIdFederationData": {
    "federatedProvider": "arn:aws:iam::<account id>:oidc-provider/token.actions.githubusercontent.com",
    "attributes": {}
}

That is, "attributes" maps from string to an object or another map, not from string to string.

This caused most of the column values to be blank for records whose webIdFederationData contained

"attributes": {}

as above. No error was thrown.

When I deleted the line

webIdFederationData:map<string,string>

from my CREATE EXTERNAL TABLE query, the column values were no longer blank.

Please update the docs to say what the data type for webIdFederationData should be, instead of map<string,string>.

I have a role that GitHub Actions assumes, following these docs:

https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services

I deployed my CloudFormation stack in a dev environment using that role, where the role had admin access.

I am trying to see what actions the role actually took, so that in my test and prod environments, I can grant only the privileges needed, instead of full admin access.

[douglasnaphas]

Looks like it's the CloudFormation/CDK role that actually creates the resources, not the role that GitHub Actions assumes. That's good. It means I have many fewer privileges to identify and grant to the role I create for GitHub Actions to assume.

In Athena,

SELECT useridentity.sessionContext.sessionIssuer.arn AS principal, resources
FROM "default"."cloudtrail_logs_aws_controltower_logs_636074447003_us_east_1_org_part_acct4"
WHERE
    account = '287213416300' AND
    eventname = 'CreateTable' AND
    eventsource = 'dynamodb.amazonaws.com'
;

returned

#   principal   resources
1   arn:aws:iam::287213416300:role/cdk-hnb659fds-cfn-exec-role-287213416300-us-east-1   [{arn=arn:aws:dynamodb:us-east-1:287213416300:table/DouglasnaphasMadliberation558-oidc-webapp-SedersTable7FFD9727-FHSICUX5WVWN, accountid=287213416300, type=AWS::DynamoDB::Table}]
2   arn:aws:iam::287213416300:role/cdk-hnb659fds-cfn-exec-role-287213416300-us-east-1   [{arn=arn:aws:dynamodb:us-east-1:287213416300:table/DouglasnaphasMadliberation558-oidc-webapp-SedersTable7FFD9727-B0AGPTA43P4A, accountid=287213416300, type=AWS::DynamoDB::Table}]
3   arn:aws:iam::287213416300:role/cdk-hnb659fds-cfn-exec-role-287213416300-us-east-1   [{arn=arn:aws:dynamodb:us-east-1:287213416300:table/DouglasnaphasMadliberation558-oidc-webapp-SedersTable7FFD9727-QEC493XB1Y1W, accountid=287213416300, type=AWS::DynamoDB::Table}]

[douglasnaphas]

TODO: Check out the trust policy on the CDK cfn-exec-role shown in my previous comment.

[douglasnaphas]

I'm going to exclude the Role-related IAM privileges, since I'm going to stop bootstrapping in Actions.

SELECT eventname,
    json_extract(requestparameters, '$.roleName') as rolename
FROM "default"."cloudtrail_logs_aws_controltower_logs_636074447003_us_east_1_org_part_acct4"
WHERE eventname IN (
        'GetRole',
        'ListAttachedRolePolicies',
        'ListRolePolicies',
        'PutRolePolicy'
    )
    AND useridentity.sessionContext.sessionIssuer.arn = 'arn:aws:iam::287213416300:role/github-actions@madliberation';

#   eventname   rolename
1   GetRole "cdk-hnb659fds-cfn-exec-role-287213416300-us-east-1"
2   ListAttachedRolePolicies    "cdk-hnb659fds-cfn-exec-role-287213416300-us-east-1"
3   ListRolePolicies    "cdk-hnb659fds-cfn-exec-role-287213416300-us-east-1"
4   PutRolePolicy   "cdk-hnb659fds-deploy-role-287213416300-us-east-1"

[douglasnaphas]

Formatting the list of allowed actions

$ cat gh-role-perms.txt | gawk 'NR != 1 && $4 != "iam.amazonaws.com" {printf("\"%s:%s\",\n", gensub(/[.].*$/, "", "g", $4), $5);}'
"cloudformation:CreateChangeSet",
"cloudformation:DeleteChangeSet",
"cloudformation:DescribeChangeSet",
"cloudformation:DescribeStackEvents",
"cloudformation:DescribeStacks",
"cloudformation:ExecuteChangeSet",
"cloudformation:GetTemplate",
"cognito-idp:AdminCreateUser",
"cognito-idp:DescribeUserPool",
"s3:GetAccelerateConfiguration",
"s3:GetBucketAnalyticsConfiguration",
"s3:GetBucketCors",
"s3:GetBucketEncryption",
"s3:GetBucketIntelligentTieringConfiguration",
"s3:GetBucketInventoryConfiguration",
"s3:GetBucketLifecycle",
"s3:GetBucketLogging",
"s3:GetBucketMetricsConfiguration",
"s3:GetBucketNotification",
"s3:GetBucketObjectLockConfiguration",
"s3:GetBucketOwnershipControls",
"s3:GetBucketPublicAccessBlock",
"s3:GetBucketReplication",
"s3:GetBucketTagging",
"s3:GetBucketVersioning",
"s3:GetBucketWebsite",
"s3:PutBucketTagging",
"sns:GetSMSSandboxAccountStatus",
"ssm:GetParameters",
"ssm:PutParameter",
"sts:AssumeRole",
"sts:GetCallerIdentity",

Not sure if all these actions from CloudTrail match the names of the related IAM action.

[douglasnaphas]

Done. Mad Liberation's main build succeeds with no saved keys.

[douglasnaphas]

I deleted all IAM users in the test and prod2 accounts.