Closed dleavitt closed 2 weeks ago
Reproducible using customer provided code (for reproduction, new line character should be added when executing git commit -m "
from console, pressing <<ENTER>>
before closing message with "
character).
If we tweak Lambda code to below (notice additional line console.log('PAYLOAD:\\n' + payload);
):
const actionLambda = `
const { CodePipeline } = require("@aws-sdk/client-codepipeline");
async function onEvent(event, context, callback) {
const codepipeline = new CodePipeline();
const job = event["CodePipeline.job"];
const payload = job.data.actionConfiguration.configuration.UserParameters;
console.log('PAYLOAD:\\n' + payload);
try {
const r = JSON.parse(payload);
console.log(r);
await codepipeline.putJobSuccessResult({ jobId: job.id });
callback(null);
} catch (ex) {
console.error(job.data.actionConfiguration.configuration);
await codepipeline.putJobFailureResult({
jobId: job.id,
failureDetails: {
type: "JobFailed",
message: ex.message,
},
});
callback(ex);
}
}
module.exports = { onEvent };
`;
It logs below in CloudWatch when Lambda is executed:
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| timestamp | message |
|---------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 1724696935703 | INIT_START Runtime Version: nodejs:20.v27 Runtime Version ARN: arn:aws:lambda:us-east-2::runtime:672d5a3e06f81d120c089c5414b05186d7b4098504797c766bde2459847f38bc |
| 1724696936018 | START RequestId: 021a6b35-5b66-4a7d-958a-25a8ff60abd1 Version: $LATEST |
| 1724696936089 | 2024-08-26T18:28:56.089Z 021a6b35-5b66-4a7d-958a-25a8ff60abd1 INFO PAYLOAD: {"msg":"Testing newline in commit message.","works":"a\nb"} |
| 1724696936111 | 2024-08-26T18:28:56.111Z 021a6b35-5b66-4a7d-958a-25a8ff60abd1 ERROR { FunctionName: 'CdkReproStack-ActionLambda1304E65B-waXCZ9u3WpLq', UserParameters: '{"msg":"Testing newline in\ncommit message.","works":"a\\nb"}' } |
| 1724696937169 | 2024-08-26T18:28:57.169Z 021a6b35-5b66-4a7d-958a-25a8ff60abd1 ERROR Invoke Error {"errorType":"SyntaxError","errorMessage":"Bad control character in string literal in JSON at position 26","stack":["SyntaxError: Bad control character in string literal in JSON at position 26"," at JSON.parse (<anonymous>)"," at Runtime.onEvent [as handler] (/var/task/index.js:12:20)"," at Runtime.handleOnceNonStreaming (file:///var/runtime/index.mjs:1173:29)"]} |
| 1724696937190 | END RequestId: 021a6b35-5b66-4a7d-958a-25a8ff60abd1 |
| 1724696937190 | REPORT RequestId: 021a6b35-5b66-4a7d-958a-25a8ff60abd1 Duration: 1171.10 ms Billed Duration: 1172 ms Memory Size: 128 MB Max Memory Used: 83 MB Init Duration: 313.07 ms |
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Notice that UserParamerters
doesn't have escaped \n
in msg
parameter.
The synthesized template has AWS::CodePipeline::Pipeline
resource as shown below:
...
"PipelineC660917D": {
"Type": "AWS::CodePipeline::Pipeline",
"Properties": {
"ArtifactStore": {
"Location": {
"Ref": "PipelineArtifactsBucket22248F97"
},
"Type": "S3"
},
"PipelineType": "V2",
"RoleArn": {
"Fn::GetAtt": [
"PipelineRoleD68726F7",
"Arn"
]
},
"Stages": [
{
"Actions": [
{
"ActionTypeId": {
"Category": "Source",
"Owner": "AWS",
"Provider": "CodeStarSourceConnection",
"Version": "1"
},
"Configuration": {
"ConnectionArn": "arn:aws:codestar-connections:us-east-2:139480602983:connection/629db161-8f01-4eb4-b763-77eef724298c",
"FullRepositoryId": "ashishdhingra/testrepo",
"BranchName": "master"
},
"Name": "GitSource",
"Namespace": "Source_GitSource_NS",
"OutputArtifacts": [
{
"Name": "Artifact_Source_GitSource"
}
],
"RoleArn": {
"Fn::GetAtt": [
"PipelineSourceGitSourceCodePipelineActionRole42BAD9EA",
"Arn"
]
},
"RunOrder": 1
}
],
"Name": "Source"
},
{
"Actions": [
{
"ActionTypeId": {
"Category": "Invoke",
"Owner": "AWS",
"Provider": "Lambda",
"Version": "1"
},
"Configuration": {
"FunctionName": {
"Ref": "ActionLambda1304E65B"
},
"UserParameters": "{\"msg\":\"#{Source_GitSource_NS.CommitMessage}\",\"works\":\"a\\nb\"}"
},
"Name": "LambdaInvoke",
"RoleArn": {
"Fn::GetAtt": [
"PipelineRunLambdaInvokeCodePipelineActionRole851BD08E",
"Arn"
]
},
"RunOrder": 1
}
],
"Name": "Run"
}
]
},
"DependsOn": [
"PipelineRoleDefaultPolicyC7A05455",
"PipelineRoleD68726F7"
],
"Metadata": {
"aws:cdk:path": "CdkReproStack/Pipeline/Resource"
}
},
...
The UserParameters
just has reference to #{Source_GitSource_NS.CommitMessage}
. The expression is generated via variableExpression() helper method (also see CodePipeline: Variables reference).
@dleavitt Thanks for opening the issue. I came across the AWS post https://repost.aws/questions/QU2jfrtUlyQqyQ5YuUSgIEow/unexpected-codepipline-variable-resolution-error-bug and it appears that CodePipeline doesn't escape special characters at this moment. The only workaround is to preprocess variables individually in a Lambda. It's a CodePipeline limitation, not the CDK limitation, hence, we should not deprecate commitMessage
property. Adding warning could be an option, but it could get out-of-sync in case CodePipeline decides to handle escape characters.
Please review above findings and confirm if we could close the issue.
Thanks, Ashish
This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.
Comments on closed issues and PRs are hard for our team to see. If you need help, please open a new issue that references this one.
Describe the bug
A CodePipeline AWS Lambda Invoke action will not receive valid JSON if passed a variable from a CodeConnection action that has a space or other special character in it.
More specifically, if your git commit message has a newline (or quote, etc) in it and you pass it as a UserParameter to a
LambdaInvokeAction
action, your lambda will crash when it tries to deserialize it.This is going to affect pretty much anyone who's trying to use a Github commit message in a Pipeline like this, since Github's merge commit message have newlines in them.
This likely has the same root cause as #8458 but is maybe more egregious since
LambdaInvokeAction
specifically does JSON serialization in the CDK.Regression Issue
Last Known Working CDK Version
No response
Expected Behavior
I'd expect the variable expansion to be JSON-safe or there to be some way to make it JSON-safe.
In general I'd expect output variables from actions to be usable as inputs to other actions, which currently doesn't seem to be the case.
Current Behavior
The variable expansion isn't valid JSON and as far as I know there's no way to get it to be. When you try to parse the UserParameters, your lambda will crash with a message like this:
Bad control character in string literal in JSON at position
Reproduction Steps
Create or find a git repo and connect it to AWS via a CodeConnection.
Make a couple of commits to the repo, one with a newline in the commit message, one without.
Deploy the stack below, passing it the arn of the code connection and the owner/name of the Git repo, like so:
CONNECTION_ARN=arn:aws:codeconnections:regions:account:connection/id GIT_REPO=owner/repo cdk deploy
Create a pipeline release of a commit where the commit message has a newline in the name. The release will fail.
Possible Solution
#{XYZ.CommitMessage.json}
or#{XYZ.quoted}
or even#{XYZ.CommitMessageQuoted}
or something. Doubt this is fixable at the CDK level.commitMessage
variable especially seems like a footgun due to lack of escaping. Consider deprecating it or putting a warning in a docblock.Additional Information/Context
No response
CDK CLI Version
2.150.0 (build 3f93027)
Framework Version
2.150.0
Node.js Version
20.11.1
OS
MacOS 14.3
Language
TypeScript
Language Version
TypeScript (5.5.3)
Other information
No response