Open simond opened 3 months ago
Findings:
DagProcessingLogs
, SchedulerLogs
, TaskLogs
, WebserverLogs
and WorkerLogs
are of type ModuleLoggingConfiguration.ModuleLoggingConfiguration, LogLevel
is optional.
LogLevel
Defines the Apache Airflow logs to send for the log type (e.g. DagProcessingLogs) to CloudWatch Logs. Valid values: CRITICAL, ERROR, WARNING, INFO.
Required: No
Type: String
Allowed values: CRITICAL | ERROR | WARNING | INFO | DEBUG
Update requires: [No interruption](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-updating-stacks-update-behaviors.html#update-no-interrupt)
The CloudFormation resource spec here also states LogLevel
as optional.
However, deploying the following CDK stack:
import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as mwaa from 'aws-cdk-lib/aws-mwaa';
export class Issue30645Stack extends cdk.Stack {
constructor(scope: Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
const mwaaEnvironment = new mwaa.CfnEnvironment(this, "MwaaEnvironment", {
name: 'MyMwaaEnvironment',
loggingConfiguration: {
taskLogs: {enabled: false, logLevel: 'INFO'},
webserverLogs: {enabled: false},
schedulerLogs: {enabled: false},
workerLogs: {enabled: false},
dagProcessingLogs: {enabled: false},
}
});
}
}
gives the below validation error from CloudFormation:
11:23:22 AM | CREATE_FAILED | AWS::MWAA::Environment | MwaaEnvironment
Resource handler returned message: "Invalid request provided: 8 validation errors detected: Value null at 'dagS3Path' failed to satisfy constraint: Member must not be null; Value nu
ll at 'loggingConfiguration.schedulerLogs.logLevel' failed to satisfy constraint: Member must not be null; Value null at 'loggingConfiguration.dagProcessingLogs.logLevel' failed to
satisfy constraint: Member must not be null; Value null at 'loggingConfiguration.webserverLogs.logLevel' failed to satisfy constraint: Member must not be null; Value null at 'loggin
gConfiguration.workerLogs.logLevel' failed to satisfy constraint: Member must not be null; Value null at 'networkConfiguration' failed to satisfy constraint: Member must not be null
; Value null at 'sourceBucketArn' failed to satisfy constraint: Member must not be null; Value null at 'executionRoleArn' failed to satisfy constraint: Member must not be null (Serv
ice: Mwaa, Status Code: 400, Request ID: 8319e3d0-4320-4748-8ff1-c873cb413650)" (RequestToken: b6c7274d-c040-c02e-b1d6-f43c071438d3, HandlerErrorCode: InvalidRequest)
❌ Issue30645Stack failed: Error: The stack named Issue30645Stack failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_COMPLETE: Resource handler returned message: "Invalid request provided: 8 validation errors detected: Value null at 'dagS3Path' failed to satisfy constraint: Member must not be null; Value null at 'loggingConfiguration.schedulerLogs.logLevel' failed to satisfy constraint: Member must not be null; Value null at 'loggingConfiguration.dagProcessingLogs.logLevel' failed to satisfy constraint: Member must not be null; Value null at 'loggingConfiguration.webserverLogs.logLevel' failed to satisfy constraint: Member must not be null; Value null at 'loggingConfiguration.workerLogs.logLevel' failed to satisfy constraint: Member must not be null; Value null at 'networkConfiguration' failed to satisfy constraint: Member must not be null; Value null at 'sourceBucketArn' failed to satisfy constraint: Member must not be null; Value null at 'executionRoleArn' failed to satisfy constraint: Member must not be null (Service: Mwaa, Status Code: 400, Request ID: 8319e3d0-4320-4748-8ff1-c873cb413650)" (RequestToken: b6c7274d-c040-c02e-b1d6-f43c071438d3, HandlerErrorCode: InvalidRequest)
at FullCloudFormationDeployment.monitorDeployment (/usr/local/lib/node_modules/aws-cdk/lib/index.js:451:10568)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async Object.deployStack2 [as deployStack] (/usr/local/lib/node_modules/aws-cdk/lib/index.js:454:199716)
at async /usr/local/lib/node_modules/aws-cdk/lib/index.js:454:181438
❌ Deployment failed: Error: The stack named Issue30645Stack failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_COMPLETE: Resource handler returned message: "Invalid request provided: 8 validation errors detected: Value null at 'dagS3Path' failed to satisfy constraint: Member must not be null; Value null at 'loggingConfiguration.schedulerLogs.logLevel' failed to satisfy constraint: Member must not be null; Value null at 'loggingConfiguration.dagProcessingLogs.logLevel' failed to satisfy constraint: Member must not be null; Value null at 'loggingConfiguration.webserverLogs.logLevel' failed to satisfy constraint: Member must not be null; Value null at 'loggingConfiguration.workerLogs.logLevel' failed to satisfy constraint: Member must not be null; Value null at 'networkConfiguration' failed to satisfy constraint: Member must not be null; Value null at 'sourceBucketArn' failed to satisfy constraint: Member must not be null; Value null at 'executionRoleArn' failed to satisfy constraint: Member must not be null (Service: Mwaa, Status Code: 400, Request ID: 8319e3d0-4320-4748-8ff1-c873cb413650)" (RequestToken: b6c7274d-c040-c02e-b1d6-f43c071438d3, HandlerErrorCode: InvalidRequest)
at FullCloudFormationDeployment.monitorDeployment (/usr/local/lib/node_modules/aws-cdk/lib/index.js:451:10568)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async Object.deployStack2 [as deployStack] (/usr/local/lib/node_modules/aws-cdk/lib/index.js:454:199716)
at async /usr/local/lib/node_modules/aws-cdk/lib/index.js:454:181438
The stack named Issue30645Stack failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_COMPLETE: Resource handler returned message: "Invalid request provided: 8 validation errors detected: Value null at 'dagS3Path' failed to satisfy constraint: Member must not be null; Value null at 'loggingConfiguration.schedulerLogs.logLevel' failed to satisfy constraint: Member must not be null; Value null at 'loggingConfiguration.dagProcessingLogs.logLevel' failed to satisfy constraint: Member must not be null; Value null at 'loggingConfiguration.webserverLogs.logLevel' failed to satisfy constraint: Member must not be null; Value null at 'loggingConfiguration.workerLogs.logLevel' failed to satisfy constraint: Member must not be null; Value null at 'networkConfiguration' failed to satisfy constraint: Member must not be null; Value null at 'sourceBucketArn' failed to satisfy constraint: Member must not be null; Value null at 'executionRoleArn' failed to satisfy constraint: Member must not be null (Service: Mwaa, Status Code: 400, Request ID: 8319e3d0-4320-4748-8ff1-c873cb413650)" (RequestToken: b6c7274d-c040-c02e-b1d6-f43c071438d3, HandlerErrorCode: InvalidRequest)
Same goes for SourceBucketArn
which is specified as not required at AWS::MWAA::Environment.
Most likely an issue with CloudFormation resource specification.
Internal tracking ticket: P137161118
Another internal ticket (from above ticket): P137427298
Describe the bug
When trying to instantiate a new
CfnEnvironment
with logging configuration set to the following:I get an error when running CDK deploy:
The error is due to the
logLevel
attribute not being passed into the logging options. The following fixes the issue:The issue is that that
logLevel
is not a required parameter according to the TypeScript interface:Expected Behavior
CDK should either:
logLevel
(since I've disabled logging anyway) and deploy the instance of MWAA orlogLevel
Current Behavior
Error message asking me to provide the
logLevel
Reproduction Steps
Possible Solution
Provide a
logLevel
Additional Information/Context
No response
CDK CLI Version
2.145.0 (build fdf53ba)
Framework Version
No response
Node.js Version
v22.0.0
OS
osx
Language
TypeScript
Language Version
v10.9.2
Other information
No response