aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.72k stars 3.94k forks source link

aws_glue: CfnTrigger.predicate.conditions (ConditionProperty) has required fields marked as optional. #24575

Closed jonabemmbo closed 1 year ago

jonabemmbo commented 1 year ago

Describe the bug

When writing a glue trigger:

 const etlS3JobTrigger = new glue.CfnTrigger(
      this,
      `DataEng-Trigger-ETLS3Job-${props.env}`,
      {
        name: `etl-s3-job-${props.env}`,
        type: "CONDITIONAL",
        predicate: {
          logical: "ANY",
          conditions: [
            {
              crawlerName: dynamoCrawler.name,
              logicalOperator: "EQUALS",
              state: "SUCCEEDED",
            },
          ],
        },
        startOnCreation: true,
        workflowName: glueWorkflow.name,
        actions: [
          {
            jobName: glueDynamoToS3Job.name,
          },
        ],
      }
    );

I get the following error:

State cannot be null or empty, choose from one of the following crawler state: [SUCCEEDED, FAILED, CANCELLED] (Service: AWSGlue; Status Code: 400; Error Code: InvalidInputException; Request ID: 87aad376-3
c08-44a4-a52f-4e5bd7d54fbb; Proxy: null)

But the documentation for the latest version (2.68.0) says it's optional. It also happens with logicalOperator.

Btw, if you can, please make clear the difference between state and crawlerState in the docs. It's not clear.

Expected Behavior

When deploying, there should not be an error if the parameter is optional. Or otherwise, make the parameter required.

Current Behavior

When deploying, I got the following error:

State cannot be null or empty, choose from one of the following crawler state: [SUCCEEDED, FAILED, CANCELLED] (Service: AWSGlue; Status Code: 400; Error Code: InvalidInputException; Request ID: 87aad376-3
c08-44a4-a52f-4e5bd7d54fbb; Proxy: null)

Reproduction Steps

 const etlS3JobTrigger = new glue.CfnTrigger(
      this,
      `DataEng-Trigger-ETLS3Job-${props.env}`,
      {
        name: `etl-s3-job-${props.env}`,
        type: "CONDITIONAL",
        predicate: {
          logical: "ANY",
          conditions: [
            {
              crawlerName: dynamoCrawler.name,
              logicalOperator: "EQUALS",
              state: "SUCCEEDED",
            },
          ],
        },
        startOnCreation: true,
        workflowName: glueWorkflow.name,
        actions: [
          {
            jobName: glueDynamoToS3Job.name,
          },
        ],
      }
    );

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.68.0

Framework Version

No response

Node.js Version

18.12.1

OS

Mac OS Ventura 13.0.1

Language

Typescript

Language Version

No response

Other information

No response

pahud commented 1 year ago

Hi

According to the CFN doc, state is optional. Looking at your code:

 const etlS3JobTrigger = new glue.CfnTrigger(
      this,
      `DataEng-Trigger-ETLS3Job-${props.env}`,
      {
        name: `etl-s3-job-${props.env}`,
        type: "CONDITIONAL",
        predicate: {
          logical: "ANY",
          conditions: [
            {
              crawlerName: dynamoCrawler.name,
              logicalOperator: "EQUALS",
              state: "SUCCEEDED",
            },
          ],
        },
        startOnCreation: true,
        workflowName: glueWorkflow.name,
        actions: [
          {
            jobName: glueDynamoToS3Job.name,
          },
        ],
      }
    );

You have provided state: "SUCCEEDED" already. Did you get the State cannot be null or empty error even when you specified state: "SUCCEEDED"?

github-actions[bot] commented 1 year ago

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

birkenbach commented 1 year ago

I guess at is a crawler you must use the crawl_state and not state.


const etlS3JobTrigger = new glue.CfnTrigger(
      this,
      `DataEng-Trigger-ETLS3Job-${props.env}`,
      {
        name: `etl-s3-job-${props.env}`,
        type: "CONDITIONAL",
        predicate: {
          logical: "ANY",
          conditions: [
            {
              crawlerName: dynamoCrawler.name,
              logicalOperator: "EQUALS",
              crawl_state: "SUCCEEDED",
            },
          ],
        },
        startOnCreation: true,
        workflowName: glueWorkflow.name,
        actions: [
          {
            jobName: glueDynamoToS3Job.name,
          },
        ],
      }
    );```