Open ghost opened 3 weeks ago
grantRead essentially only grant GetSecretValue
and DescribeSecret
to the secret. Obviously you will need BatchGetSecretValue
.
Before we have a new static method like grantBatchRead(), one option you could have though is to addToPrincipalOrResource() by yourself:
PoC
export class MyStack extends cdk.Stack {
constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
super(scope, id, props);
// Create a secret in Secrets Manager
const secret = new secretsmanager.Secret(this, 'MySecret', {
secretName: 'my-secret',
generateSecretString: {
secretStringTemplate: JSON.stringify({ username: 'myuser' }),
generateStringKey: 'password',
},
});
// Create a Lambda function
const lambdaFunction = new lambda.Function(this, 'MyLambdaFunction', {
runtime: lambda.Runtime.NODEJS_LATEST,
handler: 'index.handler',
code: lambda.Code.fromAsset('lambda'),
});
// Grant the Lambda function's role access to the secret
iam.Grant.addToPrincipalOrResource({
grantee: lambdaFunction.role!,
actions: ['secretsmanager:BatchGetSecretValue'],
resource: secret,
resourceArns: [secret.secretArn],
})
// secret.grantRead(lambdaFunction.role!);
}
}
On synth you should see a new principal policy attached to your lambda service role
MyLambdaFunctionServiceRoleDefaultPolicy23555F9E:
Type: AWS::IAM::Policy
Properties:
PolicyDocument:
Statement:
- Action: secretsmanager:BatchGetSecretValue
Effect: Allow
Resource:
Ref: MySecret8FE80B51
Version: "2012-10-17"
PolicyName: MyLambdaFunctionServiceRoleDefaultPolicy23555F9E
Roles:
- Ref: MyLambdaFunctionServiceRole313A4D46
And if you check that policy from the console
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "secretsmanager:BatchGetSecretValue",
"Resource": "arn:aws:secretsmanager:us-east-1:ACCOUNT_ID:secret:my-secret-Z0p63c",
"Effect": "Allow"
}
]
}
Now, if you need to "import" a secret using the from* method. You should always use fromSecretCompleteArn() when possible because that would end with the -
and 6 chars.
// Create a secret in Secrets Manager
const secret = new secretsmanager.Secret(this, 'MySecret', {
secretName: 'my-secret',
generateSecretString: {
secretStringTemplate: JSON.stringify({ username: 'myuser' }),
generateStringKey: 'password',
},
});
const importedSecret = secretsmanager.Secret.fromSecretCompleteArn(this, 'ImportedSecret', secret.secretArn);
And cdk deploy
and verify the policy from the IAM console. Make sure the trailing -XXXXXX
is included then you should be good.
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "secretsmanager:BatchGetSecretValue",
"Resource": "arn:aws:secretsmanager:us-east-1:ACCOUNT_ID:secret:my-secret-Z0p63c",
"Effect": "Allow"
}
]
}
This should be exactly the permission you need. Let me know if it works for you.
@pahud Thanks for the quick response!
I'm not sure if I'm missing something, but it still isn't working. Here's how I did it for my secrets:
for (let i = 0; i < MY_SECRET_NAMES.length; i++)
{
const secretName = MY_SECRET_NAMES[i]
const secret = Secret.fromSecretNameV2(
this,
`${secretName}Secret`,
secretName
)
const secretFullArn = Secret.fromSecretCompleteArn(
this,
`${secretName}SecretFullArn`,
secret.secretArn
)
secretFullArn.grantRead(myLambda)
Grant.addToPrincipalOrResource({
grantee: myLambda,
actions: ['secretsmanager:BatchGetSecretValue'],
resource: secretFullArn,
// Also tried [secretFullArn.secretArn]
resourceArns: [secretFullArn.secretFullArn!]
})
}
I can see in the Lambda > Configuration section of the Console that each of the following actions is present for each secret resource:
Allow: secretsmanager:BatchGetSecretValue
Allow: secretsmanager:DescribeSecret
Allow: secretsmanager:GetSecretValue
But the ARNs don't have the trailing characters. They just look like this:
arn:aws:secretsmanager:region:account:secret:SecretName
As far as I can tell I followed your example. Did I miss something? I tested and it doesn't even work with retrieving the secrets individually now. The ARNs did at one point show either the full ARN or with the -??????
suffix. But now, no matter what, the ARNs in the console don't have that suffix.
If I make one small change:
for (let i = 0; i < MY_SECRET_NAMES.length; i++)
{
const secretName = MY_SECRET_NAMES[i]
const secret = Secret.fromSecretNameV2(
this,
`${secretName}Secret`,
secretName
)
const secretFullArn = Secret.fromSecretCompleteArn(
this,
`${secretName}SecretFullArn`,
secret.secretArn
)
// Changed from `secretFullArn.grantRead(myLambda)`.
secret.grantRead(myLambda)
// Also changed from `secretFullArn` to `secret` here.
Grant.addToPrincipalOrResource({
grantee: myLambda,
actions: ['secretsmanager:BatchGetSecretValue'],
resource: secret,
resourceArns: [secret.secretArn]
})
}
Specifically, I did not use secretFullArn
and instead used secret
in both grantRead
and addToPrincipalOrResource
.
The results:
Allow: secretsmanager:DescribeSecret
- Now uses the full ARN with the -??????
suffix.Allow: secretsmanager:GetSecretValue
- Now uses the full ARN with the -??????
suffix.Allow: secretsmanager:BatchGetSecretValue
- Still uses the incomplete ARN and results in an error.I'm not really sure what's going on. I can retrieve the secrets individually using secret.grantRead(myLambda)
, but I don't want to do that since making multiple network calls when I could just make one is going to add up.
Let's say I have a secret created out of CDK or from another stack but I know its full ARN as arn:aws:secretsmanager:us-east-1:123456789012:secret:my-secret-Z0p63c
export class MyStack extends cdk.Stack {
constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
super(scope, id, props);
const existingSecretArn = 'arn:aws:secretsmanager:us-east-1:123456789012:secret:my-secret-Z0p63c';
const importedSecret = secretsmanager.Secret.fromSecretCompleteArn(this, 'ImportedSecret', existingSecretArn);
// Create a random Lambda function
const lambdaFunction = new lambda.Function(this, 'MyLambdaFunction', {
runtime: lambda.Runtime.NODEJS_LATEST,
handler: 'index.handler',
code: lambda.Code.fromAsset('lambda'),
});
// Grant the common read permissions to the lambda role
importedSecret.grantRead(lambdaFunction.role!);
// Grant the Lambda function's role access to the secret
iam.Grant.addToPrincipalOrResource({
grantee: lambdaFunction.role!,
actions: ['secretsmanager:BatchGetSecretValue'],
resource: importedSecret,
resourceArns: [importedSecret.secretArn],
})
}
}
Your lambda service role should have policies like this:
Is this something you expect?
Now, if you only know the secret name and you'd like the lambda role to access the ARN with a wildcard trailing chars(-?????) you could do it this way.
export class MyStack extends cdk.Stack {
constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
super(scope, id, props);
const importedSecret = secretsmanager.Secret.fromSecretNameV2(this, 'ImportedSecret', 'my-secret');
// Create a random Lambda function
const lambdaFunction = new lambda.Function(this, 'MyLambdaFunction', {
runtime: lambda.Runtime.NODEJS_LATEST,
handler: 'index.handler',
code: lambda.Code.fromAsset('lambda'),
});
// Grant the common read permissions to the lambda role
importedSecret.grantRead(lambdaFunction.role!);
// Grant the Lambda function's role access to the secret
iam.Grant.addToPrincipalOrResource({
grantee: lambdaFunction.role!,
actions: ['secretsmanager:BatchGetSecretValue'],
resource: importedSecret,
resourceArns: [ `${importedSecret.secretArn}-??????`],
})
}
}
You get this in the policy with 2 statements:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"secretsmanager:DescribeSecret",
"secretsmanager:GetSecretValue"
],
"Resource": "arn:aws:secretsmanager:us-east-1:ACCOUNT_ID:secret:my-secret-??????",
"Effect": "Allow"
},
{
"Action": "secretsmanager:BatchGetSecretValue",
"Resource": "arn:aws:secretsmanager:us-east-1:ACCOUNT_ID:secret:my-secret-??????",
"Effect": "Allow"
}
]
}
This should work too.
Or alternatively to bake the policy and statements all by yourself:
export class MyStack extends cdk.Stack {
constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
super(scope, id, props);
const importedSecret = secretsmanager.Secret.fromSecretNameV2(this, 'ImportedSecret', 'my-secret');
// Create a random Lambda function
const lambdaFunction = new lambda.Function(this, 'MyLambdaFunction', {
runtime: lambda.Runtime.NODEJS_LATEST,
handler: 'index.handler',
code: lambda.Code.fromAsset('lambda'),
});
lambdaFunction.addToRolePolicy(new iam.PolicyStatement({
actions: [
"secretsmanager:DescribeSecret",
"secretsmanager:GetSecretValue",
"secretsmanager:BatchGetSecretValue"
],
resources: [`${importedSecret.secretArn}-??????`],
}))
}
}
You will get the best result:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"secretsmanager:BatchGetSecretValue",
"secretsmanager:DescribeSecret",
"secretsmanager:GetSecretValue"
],
"Resource": "arn:aws:secretsmanager:us-east-1:ACCOUNT_ID:secret:my-secret-??????",
"Effect": "Allow"
}
]
}
I hope it clarifies. Let me know if it works for you.
Please note I am guessing you might use BatchGetSecretValue
for multiple known secret names
export class MyStack extends cdk.Stack {
constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
super(scope, id, props);
const secretNames = ['foo', 'bar']
// make it an array of all imported secrets
const importedSecrets = secretNames.map((secretName) => secretsmanager.Secret.fromSecretNameV2(this, `ImportedSecret-${secretName}`, secretName));
// Create a random Lambda function
const lambdaFunction = new lambda.Function(this, 'MyLambdaFunction', {
runtime: lambda.Runtime.NODEJS_LATEST,
handler: 'index.handler',
code: lambda.Code.fromAsset('lambda'),
});
lambdaFunction.addToRolePolicy(new iam.PolicyStatement({
actions: [
"secretsmanager:DescribeSecret",
"secretsmanager:GetSecretValue",
"secretsmanager:BatchGetSecretValue"
],
// generate resources from importedSecrets
resources: importedSecrets.map(secret => `${secret.secretArn}-??????`),
}))
}
}
You got
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"secretsmanager:BatchGetSecretValue",
"secretsmanager:DescribeSecret",
"secretsmanager:GetSecretValue"
],
"Resource": [
"arn:aws:secretsmanager:us-east-1:ACCOUNT_ID:secret:bar-??????",
"arn:aws:secretsmanager:us-east-1:ACCOUNT_ID:secret:foo-??????"
],
"Effect": "Allow"
}
]
}
I guess this is what you want?
Correct, I'm attempting to retrieve multiple secrets with a known secret name at once using BatchGetSecretValue
.
I actually did attempt adding in -??????
manually earlier.
I made these changes based on your reply:
secret.grantRead(myLambda)
Grant.addToPrincipalOrResource({
grantee: myLambda,
actions: ['secretsmanager:BatchGetSecretValue'],
resource: secret,
resourceArns: [`${secret.secretArn}-??????`]
})
Now I can confirm that in the Lambda configuration, all three policies below apply to the a resource with the format: arn:aws:secretsmanager:us-east-1:ACCOUNT_ID:secret:my-secret-??????
:
Allow: secretsmanager:DescribeSecret
Allow: secretsmanager:GetSecretValue
Allow: secretsmanager:BatchGetSecretValue
Individual retrieval of secrets works as expected. The batch retrieval still fails with the same error, even though now it looks like the ARNs are correct on the Console.
I also attempted to grant permissions using the actual complete ARNs that I hardcoded into my CDK stack after copying them from the AWS Console, instead of using the secret names.
This had the same result - I can retrieve the secrets individually, but not as a batch.
The only difference is that now, in the AWS Console under Lambda > Configuration > Permissions, I see the complete ARN with -??????
replaced by the actual characters, and these three permissions associated with each ARN:
Allow: secretsmanager:DescribeSecret
Allow: secretsmanager:GetSecretValue
Allow: secretsmanager:BatchGetSecretValue
So overall, no matter what the ARN's value is at the time of deployment, Allow: secretsmanager:BatchGetSecretValue
doesn't actually have any effect when I try to retrieve the secrets.
for (let i = 0; i < LIST_OF_SECRET_COMPLETE_ARNS.length; i++)
{
const secretArn = LIST_OF_SECRET_COMPLETE_ARNS.length[i]
const secret = Secret.fromSecretCompleteArn(
this,
`FromSecretCompleteArn-${i}`,
secretArn
)
secret.grantRead(myLambda)
Grant.addToPrincipalOrResource({
grantee: myLambda,
actions: ['secretsmanager:BatchGetSecretValue'],
resource: secret,
resourceArns: [secretArn]
})
}
Just for clarity, this is how I'm attempting to retrieve the secrets by batch:
import {
BatchGetSecretValueCommand,
BatchGetSecretValueCommandInput,
BatchGetSecretValueCommandOutput,
GetSecretValueCommand,
GetSecretValueCommandInput,
GetSecretValueCommandOutput,
SecretsManagerClient,
SecretValueEntry
} from '@aws-sdk/client-secrets-manager'
const getSecretValueCommandInput: BatchGetSecretValueCommandInput = typeof nextToken !== 'undefined'
? { NextToken: nextToken }
: initialGetSecretValueCommandInput
const getSecretValueCommand = new BatchGetSecretValueCommand(getSecretValueCommandInput)
const getSecretValueCommandOutput: BatchGetSecretValueCommandOutput = await secretsManagerClient.send(getSecretValueCommand)
Did you mean you are still seeing this error?
{
"name": "AccessDeniedException",
"$fault": "client",
"$metadata": {
"httpStatusCode": 400,
"requestId": "123456",
"attempts": 1,
"totalRetryDelay": 0
},
"__type": "AccessDeniedException",
"message": "User: arn:aws:sts::123456 is not authorized to perform: secretsmanager:BatchGetSecretValue because no identity-based policy allows the secretsmanager:BatchGetSecretValue action"
}
Can you check if there's any resource policy on the existing secret? I am guessing there might be a secret policy stopping you from reading it.
% aws secretsmanager get-resource-policy --secret-id foo
{
"ARN": "arn:aws:secretsmanager:us-east-1:ACCOUNT:secret:foo-bJ0ZSv",
"Name": "foo"
}
AWS evaluates resource assess with identity-based policy and resource-based policy. If no resource-policy is defined, the identity-based policy should be enough to allow you to access with resource policy undefined.
I just wrote another PoC and it works great to me.
export class MyStack extends cdk.Stack {
constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
super(scope, id, props);
const secretNames = ['foo', 'bar']
// make it an array of all imported secrets
const importedSecrets = secretNames.map((secretName) => secretsmanager.Secret.fromSecretNameV2(this, `ImportedSecret-${secretName}`, secretName));
// Create a Lambda function that retrieves the 'foo' secret using AWS SDK v2
const lambdaFunction = new lambda.Function(this, 'MyLambdaFunction', {
runtime: lambda.Runtime.NODEJS_LATEST,
handler: 'index.handler',
code: lambda.Code.fromInline(`
const { SecretsManager } = require('@aws-sdk/client-secrets-manager');
const secretsManager = new SecretsManager();
exports.handler = async function(event, context) {
try {
const response = await secretsManager.getSecretValue({ SecretId: 'foo' });
if (response.SecretString) {
console.log('Secret value:', response.SecretString);
}
return { statusCode: 200, body: 'Secret retrieved successfully' };
} catch (error) {
console.error('Error retrieving secret:', error);
return { statusCode: 500, body: 'Error retrieving secret' };
}
};
`),
});
lambdaFunction.addToRolePolicy(new iam.PolicyStatement({
actions: [
"secretsmanager:DescribeSecret",
"secretsmanager:GetSecretValue",
"secretsmanager:BatchGetSecretValue"
],
// generate resources from importedSecrets
resources: importedSecrets.map(secret => `${secret.secretArn}-??????`),
}))
}
}
Right, I'm still seeing that AccessDeniedError
. It only throws when I try to retrieve the secrets in a batch, even though the batch-retrieval action is grouped in with the same ARN (suffixed with -??????
) in the Lambda configuration section of the Console. I can retrieve them individually without any errors.
I ran this command and saw this output for each of the secrets in question:
% aws secretsmanager get-resource-policy --secret-id foo
{
"ARN": "arn:aws:secretsmanager:us-east-1:ACCOUNT:secret:foo-bJ0ZSv",
"Name": "foo"
}
I'm not too sure what it means. I set up the secrets manually on the Console, rather than through the CDK, and I didn't change any of the default/optional fields other than entering the secret name and value. In the Overview > Resource permissions section of each secret on the Console, there is nothing.
It only throws when I try to retrieve the secrets in a batch, even though the batch-retrieval action is grouped in with the same ARN (suffixed with -??????) in the Lambda configuration section of the Console.
Can you share the minimal code snippet of that? I'd like to see how to batch retrieve them in Lambda.
Sure, here's how I'm doing it.
getSecretValueCommandInput
is either SecretIdList: string[]
or NextToken: string
.
import {
BatchGetSecretValueCommand,
BatchGetSecretValueCommandInput,
BatchGetSecretValueCommandOutput,
GetSecretValueCommand,
GetSecretValueCommandInput,
GetSecretValueCommandOutput,
SecretsManagerClient,
SecretValueEntry
} from '@aws-sdk/client-secrets-manager'
const getSecretValueCommandInput: BatchGetSecretValueCommandInput = typeof nextToken !== 'undefined'
? { NextToken: nextToken }
: initialGetSecretValueCommandInput
const getSecretValueCommand = new BatchGetSecretValueCommand(getSecretValueCommandInput)
const getSecretValueCommandOutput: BatchGetSecretValueCommandOutput = await secretsManagerClient.send(getSecretValueCommand)
FWIW, I ran into this very same issue. Had a lambda, previously executing 4 GetSecretValueCommand
which I replaced with a single BatchGetSecretValueCommand
after which I kept getting the no identity policy
error.
After some troubleshooting, I got this working. What isn't made clear from the documentation is that the BatchGetSecretValue
policy should be separate from the GetSecretValue
policy and allowed on all resources.
The official docs suggest that you need both secretsmanager:BatchGetSecretValue
permissions as well as secretsmanager:GetSecretValue
permission on the individual secrets themselves.
The docs on permissions for SecretsManager show that BatchGetSecretValue
does not require a resource type to be specified.
So in the end I got this working by changing my policy to something like
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"secretsmanager:GetSecretValue"
],
"Resource": [
"arn:aws:secretsmanager:us-east-1:<account>:secret:/<some-prefix>/*",
],
"Effect": "Allow"
},
{
"Action": [
"secretsmanager:BatchGetSecretValue",
"secretsmanager:ListSecrets"
],
"Resource": "*",
"Effect": "Allow"
}
]
}
@JBSchami
I found this blog post https://aws.amazon.com/blogs/security/how-to-use-the-batchgetsecretsvalue-api-to-improve-your-client-side-applications-with-aws-secrets-manager/
Where the provided iam policy is having
{
"Sid": "Statement2",
"Effect": "Allow",
"Action": [
"secretsmanager:ListSecrets",
"secretsmanager:BatchGetSecretValue"
],
"Resource": ["*"]
}
so I guess you are right.
@occassionally can you try it out and let me know if it works?
Describe the bug
I'm attempting to retrieve secrets using BatchGetSecretValue.
In my CDK app, I've created a
NodejsFunction
Lambda.Then, I retrieve the secrets as follows, and grant read access to the lambda:
This worked fine when I had only one secret and was retrieving it using GetSecretValueCommand.
However, now I've switched to using BatchGetSecretValueCommand, and it seems I no longer have access to the secrets.
I get this error when attempting the batch retrieval:
I've only shown one secret in the examples above, but in the actual app I'm repeating this process for the few secrets I have and granting read access for each one.
I found this documentation that mentions adding the following as a permission:
If I look at my lambda in the AWS Console, I can see that it does only have these action permissions for each secret:
Allow: secretsmanager:DescribeSecret
Allow: secretsmanager:GetSecretValue
And clearly
secretsmanager:BatchGetSecretValue
is not part of this.So I attempted the following:
And still got the same error. When I look in the AWS Console, the ARNs now don't include the final six characters. When I used
grantRead
, the ARNs ended in-??????
. Now they don't.I even attempted to add it back in like this:
And it made no difference.
Overall, I can't figure out how to retrieve secrets in a batch.
More specifically, what possible reason could there be for
grantRead
to provide access to read and retrieve a secret individually but not as part of a batch? If I usegrantReadWriteData
to grant the same lambda access to a DynamoDB table, I can see clearly in the Console that it includes batch-related actions. So why is there an exception for secrets? If there is permission to read it individually, what is the risk of reading it as part of a batch? This should be the default behavior, just like DynamoDB, or at least there should be an opt-in.I've read in the docs that using the
grant*
functions is best practice over providing specific policies like the one I attempted above. There should be an option to use agrant*
function for this. Having to specify a policy like the one attempted above is never a better option over something likegrant*
.Regression Issue
Last Known Working CDK Version
No response
Expected Behavior
I expected
grantRead
would cover batch retrieval as well as non-batch retrieval.Current Behavior
It does not cover that action and my lambda does not have permission to retrieves secrets by batch, although it has been granted read-related permissions to each secret in question.
Reproduction Steps
Please see above.
Possible Solution
grantRead
should cover batch retrieval.Additional Information/Context
No response
CDK CLI Version
2.159.1 (build c66f4e3)
Framework Version
No response
Node.js Version
v20.16.0
OS
MacOS
Language
TypeScript
Language Version
5.6.2
Other information
No response