ashishdhingra
commented
3 weeks ago @ksco92 Good morning. Could you please share the minimal CDK code to reproduce the issue, including CfnDataCatalogEncryptionSettings? I tried reproducing the issue at my end, somehow unable to reproduce the issue using the below code:
import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as s3 from 'aws-cdk-lib/aws-s3';
import * as kms from 'aws-cdk-lib/aws-kms';
import { aws_glue as glue } from 'aws-cdk-lib';
import * as glue_alpha from '@aws-cdk/aws-glue-alpha';
export class Issue30364GluekmsStack extends cdk.Stack {
constructor(scope: Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
const s3bucket = new s3.Bucket(this, 'test-glue-s3-bucket');
const kmsKey = new kms.Key(this, 'test-glue-kms-key', {
alias: 'test-glue-kms-key'
});
const cfnDataCatalogEncryptionSettings = new glue.CfnDataCatalogEncryptionSettings(this, 'MyCfnDataCatalogEncryptionSettings', {
catalogId: this.account,
dataCatalogEncryptionSettings: {
connectionPasswordEncryption: {
kmsKeyId: kmsKey.keyId,
returnConnectionPasswordEncrypted: true,
}
},
});
const glueDatabase = new glue_alpha.Database(this, 'test-glue-db', {
databaseName: 'test-glue-db',
description: 'Test Glue DB'
});
const glueS3Table = new glue_alpha.S3Table(this, 'test-glue-s3-table', {
database: glueDatabase,
columns: [{
name: 'col1',
type: glue_alpha.Schema.STRING,
}],
partitionKeys: [{
name: 'year',
type: glue_alpha.Schema.SMALL_INT,
}, {
name: 'month',
type: glue_alpha.Schema.SMALL_INT,
}],
partitionIndexes: [
{
keyNames: ['year'],
indexName: 'yearindex'
}
],
dataFormat: glue_alpha.DataFormat.JSON,
bucket: s3bucket,
enablePartitionFiltering: true,
});
}
}Thanks, Ashish
github-actions[bot]
ksco92
Describe the bug
If you make your catalog settings using this and add a KMS key:
https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_glue.CfnDataCatalogEncryptionSettings.html
And then make an S3 table with partition indices using this:
https://docs.aws.amazon.com/cdk/api/v2/docs/@aws-cdk_aws-glue-alpha.S3Table.html
The creation of the partition indices will fail. This is from a CW log of the custom resource:
And then:
If I remove the KMS key from the catalog settings, this succeeds.
Expected Behavior
If the catalog is encrypted, the custom resource should be given permissions to the catalog KMS key.
Current Behavior
The deployment fails because the role that runs the lambda function has no permissions to encrypt/decrypt the catalog key.
Reproduction Steps
In a single stack:
Possible Solution
Allow to pass a KMS key to S3Table to indicate that the catalog is encrypted and that the custom resource should have access to it.
Additional Information/Context
No response
CDK CLI Version
2.143.0
Framework Version
No response
Node.js Version
18
OS
Mac
Language
TypeScript
Language Version
No response
Other information
No response