feat: Support sfn->Lambda context injection when no Payload exists (case 1)

[lym953]

Current behavior:

In the Lambda step definition,

• If Payload.$ doesn't exist, then inject context using States.JsonMerge($$, $, false)
• We don't check if Payload exists. The current behavior when Payload exists is that we also try to inject context using States.JsonMerge($$, $, false). Then there would be an error when serverless deploy is run, saying Payload and Payload.$ can't both exist, which can be confusing to users.

What does this PR do?

Change context injection behavior so that if Payload.$ or Payload don't exist, then inject context using:

$$['Execution', 'State', 'StateMachine']

This corresponds to Case 1 in the design doc Fixing Step Function Instrumentation

Motivation

Testing Guidelines

Automated tests

Passed the existing and added tests

Manual tests

Steps

1. Create a State Machine with a Lambda step which has no Payload or Payload.$

   InvokeLambdaStep:
   Type: Task
   Resource: arn:aws:states:::lambda:invoke
   Parameters:
   FunctionName:
     Fn::GetAtt: [yiming_plugin_hello, Arn]
   # Payload.$: $
   End: true

2. Run serverless deploy.

3. Check the State Machine definition in AWS UI

Result

Context was injected as expected:

    "InvokeLambdaStep": {
      "Type": "Task",
      "Resource": "arn:aws:states:::lambda:invoke",
      "Parameters": {
        "FunctionName": "arn:aws:lambda:sa-east-1:425362996713:function:yiming-serverless-demo-dev-yiming_plugin_hello",
        "Payload.$": "$$['Execution', 'State', 'StateMachine']"
      },
      "End": true
    }

Additional Notes

Types of changes

• [ ] Bug fix
• [x] New feature
• [ ] Breaking change
• [ ] Misc (docs, refactoring, dependency upgrade, etc.)

Check all that apply

• [x] This PR's description is comprehensive
• [ ] This PR contains breaking changes that are documented in the description
• [ ] This PR introduces new APIs or parameters that are documented and unlikely to change in the foreseeable future
• [ ] This PR impacts documentation, and it has been updated (or a ticket has been logged)
• [x] This PR's changes are covered by the automated tests
• [ ] This PR collects user input/sensitive content into Datadog

[lym953]

/merge

[dd-devflow[bot]]

:steam_locomotive: MergeQueue: pull request added to the queue

The median merge time in main is 2m.

Use /merge -c to cancel this operation!