aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.65k stars 3.91k forks source link

cdk: cdk synth and cdk deploy create invalid property names s3StorageConfig and enableOnlineStore #25990

Closed BigNerd closed 1 year ago

BigNerd commented 1 year ago

Describe the bug

We are running cdk synth or cdk deploy against a python stack that defines a SageMaker feature group. The properties validation as part of the cdk deploy command fails with the following error messages:

/OfflineStoreConfig: required key [S3StorageConfig] not found

/OfflineStoreConfig: extraneous key [s3StorageConfig] is not permitted

/OnlineStoreConfig: extraneous key [enableOnlineStore] is not permitted

The generated YAML Cloudformation representation looks as follows:

Parameters:
  BootstrapVersion:
    Default: /cdk-bootstrap/hnb659fds/version
    Description: Version of the CDK Bootstrap resources in this environment, automatically
      retrieved from SSM Parameter Store. [cdk:skip]
    Type: AWS::SSM::Parameter::Value<String>
Resources:
  funicorn:
    Properties:
      EventTimeFeatureName: EVENT_TIME
      FeatureDefinitions:
      - FeatureName: VALUE
        FeatureType: String
      - FeatureName: ID
        FeatureType: Fractional
      - FeatureName: EVENT_TIME
        FeatureType: Fractional
      FeatureGroupName: f-unicorn
      OfflineStoreConfig:
        s3StorageConfig:
          s3Uri: s3://my-bucket-name/f-unicorn
      OnlineStoreConfig:
        enableOnlineStore: true
      RecordIdentifierFeatureName: ID
      RoleArn: arn:aws:iam::my-role-name
    Type: AWS::SageMaker::FeatureGroup

On the other hand the following documentation suggests that s3StorageConfig should start with a capital letter:

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-sagemaker-featuregroup-offlinestoreconfig.html

Expected Behavior

The stack should be deployed with the correct spelling of the property names.

Current Behavior

The deployment of the stack fails due to the validation error with regards to the wrong property names.

Reproduction Steps

The following code produces the described issue in the YAML output, which can be displayed by running cdk synth

#!/usr/bin/env python3
import aws_cdk as cdk

from aws_cdk import (
    Stack,
)
from aws_cdk import aws_sagemaker as sagemaker

class FeatureGroupStack(Stack):
    def __init__(self, scope, id, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)
        sagemaker.CfnFeatureGroup(
            self,
            id="id",
            event_time_feature_name="event_time",
            feature_definitions=[],
            feature_group_name="feature_group",
            record_identifier_feature_name="record_identifier",
            offline_store_config=sagemaker.CfnFeatureGroup.OfflineStoreConfigProperty(
                s3_storage_config=sagemaker.CfnFeatureGroup.S3StorageConfigProperty(
                    s3_uri="s3://foo/bar"
                )
            ),
            role_arn="role_arn"
        )

app = cdk.App()

FeatureGroupStack(
    app,
    f"FeatureGroupStack",
)

app.synth()

Possible Solution

No response

Additional Information/Context

The YAML output (abbreviated) from the code snippet supplied. The properties s3StorageConfig and s3Uri should actually start with captital s (S), we believe.

Resources:
  id:
    Type: AWS::SageMaker::FeatureGroup
    Properties:
      EventTimeFeatureName: event_time
      FeatureDefinitions: []
      FeatureGroupName: feature_group
      RecordIdentifierFeatureName: record_identifier
      OfflineStoreConfig:
        s3StorageConfig:
          s3Uri: s3://foo/bar
      RoleArn: role_arn
    Metadata:
      aws:cdk:path: FeatureGroupStack/id
  CDKMetadata:
...

CDK CLI Version

2.84.0 (build f7c792f)

Framework Version

No response

Node.js Version

18

OS

Linux Debian (bookworm), macOS Big Sur (11.6)

Language

Python

Language Version

Python 3.9

Other information

We installed the following python packages

aws-cdk-lib==2.82.0 aws-cdk.asset-awscli-v1==2.2.186 aws-cdk.asset-kubectl-v20==2.1.1 aws-cdk.asset-node-proxy-agent-v5==2.0.155 constructs==10.2.45

peterwoodworth commented 1 year ago

This OfflineStoreConfig property asks for type any, not the OfflineStoreConfigProperty. I know this is confusing, and we're looking into making sure this doesn't happen again.

See this comment for an example of how you should define something like this going forward https://github.com/aws/aws-cdk/issues/25774#issuecomment-1568940003

github-actions[bot] commented 1 year ago

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.