mongodb / awscdk-resources-mongodbatlas

MongoDB Atlas AWS CDK Resources
Apache License 2.0
33 stars 15 forks source link

[Bug]: CfnOnlineArchive clusterName property not working #286

Open lockenj opened 3 weeks ago

lockenj commented 3 weeks ago

Is there an existing issue for this?

CDK package version

v3.5.2

CFN Resource version

N/A

CFN Resource Region

us-east-2

Current Behavior

According to the example https://github.com/mongodb/awscdk-resources-mongodbatlas/blob/main/examples/l1-resources/online-archive.ts clusterName is not required however if its omitted I get this error:

` Properties validation failed for resource MongoAtlasConstructeventruntimeeventdetailB11AA3A2 with message:

: required key [ClusterName] not found`

Whats worse if I include it using the same name I created my cluster with I get an error stating the string does conform to the pattern when the cluster name I am using is perfectly fine (based on the cluster creating successfully).

Properties validation failed for resource MongoAtlasConstructeventruntimeeventdetailCEA3BD2F with message: [#/ClusterName: string [core-environment-manual-test] does not match pattern ^([a-zA-Z0-9]([a-zA-Z0-9-]){0,21}(?<!-)([\w]{0,42}))$]

AWS CDK code to reproduce the issue

/********************************************************************/
    // CREATE A NEW MONGODB ATLAS CLUSTER
    /********************************************************************/
    //see https://github.com/mongodb/awscdk-resources-mongodbatlas/blob/main/API.md#cfnclusterprops-
    const cfnClusterProps = {
      //{string}    Profile used to provide credentials information, (a secret with the cfn/atlas/profile/{Profile}, is required), if not provided default is used.
      profile: this.#_secretName,
      //{string}    Unique identifier of the project the cluster belongs to.
      projectId: this.#_atlasProjectId,
      //{string}    Human-readable label that identifies the advanced cluster.
      name: this.#_clusterName,
      //{ProcessArgs}   See https://github.com/mongodb/awscdk-resources-mongodbatlas/blob/main/API.md#processargs-.
      advancedSettings: undefined,
      //{boolean}   Flag that indicates whether the cluster can perform backups.
      backupEnabled: false,
      //{boolean}   Flag that indicates whether the cluster uses continuous cloud backups.
      pitEnabled: false,

      //TODO need to setup encryption at REST

      //{CfnClusterPropsEncryptionAtRestProvider}   Cloud service provider that manages your customer keys to provide an additional layer of encryption at rest for the cluster.
      //see https://github.com/mongodb/awscdk-resources-mongodbatlas/blob/main/API.md#cfnclusterpropsencryptionatrestprovider-
      encryptionAtRestProvider: undefined,

      //{CfnClusterPropsBiConnector}    Settings needed to configure the MongoDB Connector for Business Intelligence for this cluster.
      //see https://github.com/mongodb/awscdk-resources-mongodbatlas/blob/main/API.md#cfnclusterpropsbiconnector-
      biConnector: undefined,

      //{CfnClusterPropsLabels[]}   Collection of key-value pairs between 1 to 255 characters in length that tag and categorize the cluster.
      labels: undefined,
      //{string}    Major MongoDB version of the cluster.
      mongoDbMajorVersion: undefined, //We will default to the latest Mongo Version
      //{boolean}   Flag that indicates whether the cluster is paused or not.
      paused: false,
      //{string}    Root Certificate Authority that MongoDB Cloud cluster uses.
      rootCertType: undefined,
      //{Tag[]} List of settings that configure your cluster regions.
      tags: undefined,
      //{boolean}   Flag that indicates whether termination protection is enabled on the cluster.
      terminationProtectionEnabled: undefined,
      //{string}    Method by which the cluster maintains the MongoDB versions.

      //TODO need to decide what to do here
      versionReleaseSystem: undefined,

      //{string}    Configuration of nodes that comprise the cluster.
      clusterType: 'REPLICASET',
      //{number}    Storage capacity that the host's root volume possesses expressed in gigabytes.
      diskSizeGb: undefined,

      //{AdvancedReplicationSpec[]} List of settings that configure your cluster regions.
      //see https://github.com/mongodb/awscdk-resources-mongodbatlas/blob/main/API.md#advancedreplicationspec-
      replicationSpecs: [{
        numShards: 1,
        advancedRegionConfigs: [
          {
            regionName: regionTable.findInMap(this.#_region, 'upperRegionName'),
            electableSpecs: {
              instanceSize: sizeTable.findInMap(this.#_size, 'instanceSize'),
              nodeCount: 3,
              ebsVolumeType: "STANDARD",
            },

            priority: 7,
            providerName: sizeTable.findInMap(this.#_size, 'providerName'),
            backingProviderName: sizeTable.findInMap(this.#_size, 'backingProviderName'),
          },
        ],
      }],
    };
    //-------------------------------------
    // Cluster Backup Policy
    //-------------------------------------
    const backupPolicyProps = props?.clusterProps?.backupPolicy;
    delete props.clusterProps.backupPolicy;
    const backupProps = {};
    //enable backup policy on cluster
    if (backupPolicyProps) {
      //{boolean}   Flag that indicates whether the cluster can perform backups.
      backupProps.backupEnabled = true;

      //{boolean}   Flag that indicates whether the cluster uses continuous cloud backups.
      if (backupPolicyProps?.pitEnabled === true) {
        props.backupPolicy.pitEnabled = true;
      }
    }
    //-------------------------------------
    // Cluster
    //-------------------------------------
    this.#_atlasCluster = new CfnCluster(this, 'MongoAtlasCluster', {
      ...cfnClusterProps,
      ...backupProps,
      ...props.clusterProps,//allows user to override any props
    });
    this.#_atlasCluster.applyRemovalPolicy(cdk.RemovalPolicy.DESTROY);
    if (this.#_atlasProject) {
      this.#_atlasCluster.node.addDependency(this.#_atlasProject);
    }
    else {
      this.#_atlasCluster.node.addDependency(this.#_atlasSecret);
    }
    //-------------------------------------
    // Cluster Archive Policy
    //-------------------------------------
    if (props?.clusterProps?.archivePolicy) {
      const archives = {};
      for (const archiveProps of props.clusterProps.archivePolicy.archives) {
        const archiveName = `${archiveProps.dbName}.${archiveProps.collName}`;
        const archive = new CfnOnlineArchive(this, archiveName, {
          profile: this.#_secretName,
          projectId: this.#_atlasProjectId,
          // clusterName: this.#_clusterName,
          ...archiveProps
        });
        archive.node.addDependency(this.#_atlasCluster);
        archives[archiveName] = archive;
      }
    }

Steps To Reproduce

Use these properties to reproduce the issue... { organizationId: process.env.MONGO_ATLAS_ORGANIZATION_ID, projectId: process.env.MONGO_ATLAS_PROJECT_ID, projectName: process.env.MONGO_ATLAS_PROJECT_NAME, clusterName: 'core-env-test', secretName: 'core-env-test', dbRootUserName: 'core-env-test-root', size: 'small', projectProps: {}, clusterProps: { backupPolicy: { pitEnabled: false, }, archivePolicy: { archives: [ { collName: 'event-detail', dbName: 'event-runtime', collectionType: CfnOnlineArchivePropsCollectionType.STANDARD, criteria: { type: CriteriaViewType.CUSTOM, query: '{ __toBeArchived: true }', }, //ScheduleView Regular frequency and duration when archiving process occurs. //Below is 2am-4am EST schedule: { type: ScheduleViewType.DAILY, startHour: 6, //number Hour of the day when the when the scheduled window to run one online archive starts. startMinute: 0, //number Minute of the hour when the scheduled window to run one online archive starts. endHour: 8, //number Hour of the day when the scheduled window to run one online archive ends. endMinute: 0, //number Minute of the hour when the scheduled window to run one online archive ends. }, }, ] } }, }

cdk synth

Mappings:
  MongoAtlasConstructRegionTableC4532510:
    us-east-1:
      upperRegionName: US_EAST_1
    us-east-2:
      upperRegionName: US_EAST_2
    us-west-1:
      upperRegionName: US_WEST_1
    us-west-2:
      upperRegionName: US_WEST_2
    ap-south-1:
      upperRegionName: AP_SOUTH_1
    ap-northeast-3:
      upperRegionName: AP_NORTHEAST_3
    ap-northeast-2:
      upperRegionName: AP_NORTHEAST_2
    ap-southeast-1:
      upperRegionName: AP_SOUTHEAST_1
    ap-southeast-2:
      upperRegionName: AP_SOUTHEAST_2
    ap-northeast-1:
      upperRegionName: AP_NORTHEAST_1
    ca-central-1:
      upperRegionName: CA_CENTRAL_1
    eu-central-1:
      upperRegionName: EU_CENTRAL_1
    eu-west-1:
      upperRegionName: EU_WEST_1
    eu-west-2:
      upperRegionName: EU_WEST_2
    eu-west-3:
      upperRegionName: EU_WEST_3
    eu-north-1:
      upperRegionName: EU_NORTH_1
    sa-east-1:
      upperRegionName: SA_EAST_1
    af-south-1:
      upperRegionName: AF_SOUTH_1
    ap-east-1:
      upperRegionName: AP_EAST_1
    ap-south-2:
      upperRegionName: AP_SOUTH_2
    ap-southeast-3:
      upperRegionName: AP_SOUTHEAST_3
    ap-southeast-4:
      upperRegionName: AP_SOUTHEAST_4
    ca-west-1:
      upperRegionName: CA_WEST_1
    eu-south-1:
      upperRegionName: EU_SOUTH_1
  MongoAtlasConstructSizingTable71DF5D19:
    FREE:
      instanceSize: M0
      providerName: TENANT
      backingProviderName: AWS
    SMALL:
      instanceSize: M10
      providerName: AWS
      backingProviderName: AWS
    MEDIUM:
      instanceSize: M20
      providerName: AWS
      backingProviderName: AWS
    LARGE:
      instanceSize: M30
      providerName: AWS
      backingProviderName: AWS
    HUGE:
      instanceSize: M40
      providerName: AWS
      backingProviderName: AWS
Resources:
  MongoAtlasConstructMongoAtlasSecretEBBC38A5:
    Type: AWS::SecretsManager::Secret
    Properties:
      Description: "Public/private keys used to access the MongoDB Atlas Cloud Formation management api for organization: 65c27724d03c0b5768103c90."
      Name: cfn/atlas/profile/core-env-test
      SecretString: '{"PublicKey":"vhqsyufp","PrivateKey":"8b402f4a-3699-4613-bf85-fe2419fb2f31"}'
    UpdateReplacePolicy: Delete
    DeletionPolicy: Delete
    Metadata:
      aws:cdk:path: MongoClusterTestStack/MongoAtlasConstruct/MongoAtlasSecret/Resource
  MongoAtlasConstructMongoAtlasCluster48C24AB2:
    Type: MongoDB::Atlas::Cluster
    Properties:
      BackupEnabled: true
      ClusterType: REPLICASET
      Profile: core-env-test
      ProjectId: 65c7a364ba5030155d686ffa
      Name: core-env-test
      Paused: false
      PitEnabled: false
      ReplicationSpecs:
        - NumShards: 1
          AdvancedRegionConfigs:
            - RegionName:
                Fn::FindInMap:
                  - MongoAtlasConstructRegionTableC4532510
                  - Ref: AWS::Region
                  - upperRegionName
              BackingProviderName:
                Fn::FindInMap:
                  - MongoAtlasConstructSizingTable71DF5D19
                  - SMALL
                  - backingProviderName
              ProviderName:
                Fn::FindInMap:
                  - MongoAtlasConstructSizingTable71DF5D19
                  - SMALL
                  - providerName
              ElectableSpecs:
                EbsVolumeType: STANDARD
                InstanceSize:
                  Fn::FindInMap:
                    - MongoAtlasConstructSizingTable71DF5D19
                    - SMALL
                    - instanceSize
                NodeCount: 3
              Priority: 7
    DependsOn:
      - MongoAtlasConstructMongoAtlasSecretEBBC38A5
    UpdateReplacePolicy: Delete
    DeletionPolicy: Delete
    Metadata:
      aws:cdk:path: MongoClusterTestStack/MongoAtlasConstruct/MongoAtlasCluster
  MongoAtlasConstructeventruntimeeventdetailB11AA3A2:
    Type: MongoDB::Atlas::OnlineArchive
    Properties:
      Profile: core-env-test
      CollName: event-detail
      CollectionType: STANDARD
      Criteria:
        Type: CUSTOM
        Query: "{ __toBeArchived: true }"
      DbName: event-runtime
      ProjectId: 65c7a364ba5030155d686ffa
      Schedule:
        Type: DAILY
        EndHour: 8
        EndMinute: 0
        StartHour: 6
        StartMinute: 0
    DependsOn:
      - MongoAtlasConstructMongoAtlasCluster48C24AB2
    Metadata:
      aws:cdk:path: MongoClusterTestStack/MongoAtlasConstruct/event-runtime.event-detail
  MongoAtlasConstructMongoAtlasIpAccessList20D92B92:
    Type: MongoDB::Atlas::ProjectIpAccessList
    Properties:
      AccessList:
        - CIDRBlock: 0.0.0.0/1
          Comment: Big Hole 1
      ProjectId: 65c7a364ba5030155d686ffa
      Profile: core-env-test
    DependsOn:
      - MongoAtlasConstructMongoAtlasCluster48C24AB2
    UpdateReplacePolicy: Delete
    DeletionPolicy: Delete
    Metadata:
      aws:cdk:path: MongoClusterTestStack/MongoAtlasConstruct/MongoAtlasIpAccessList
  MongoAtlasConstructMongoAtlasDbUserSecretC02C1EF1:
    Type: AWS::SecretsManager::Secret
    Properties:
      Description: Secret (mongo-db-atlas/project/event-runtime-dev/cluster/core-env-test/root-user) used for accessing the MongoDB Atlas Cluster (core-env-test).
      GenerateSecretString:
        ExcludePunctuation: true
        GenerateStringKey: password
        IncludeSpace: false
        RequireEachIncludedType: true
        SecretStringTemplate: '{"username":"core-env-test-root"}'
      Name: mongo-db-atlas/project/event-runtime-dev/cluster/core-env-test/root-user
    UpdateReplacePolicy: Delete
    DeletionPolicy: Delete
    Metadata:
      aws:cdk:path: MongoClusterTestStack/MongoAtlasConstruct/MongoAtlasDbUserSecret/Resource
  MongoAtlasConstructMongoAtlasConnectionStringSecretCFD2F3EA:
    Type: AWS::SecretsManager::Secret
    Properties:
      Description: Mongo DB Connection String for the root user of the core-env-test cluster.
      Name: mongo-db-atlas/project/event-runtime-dev/cluster/core-env-test/root-user/mongo-connection-string
      SecretString:
        Fn::Join:
          - ""
          - - '{"MONGO_CONNECTION_STRING":"'
            - Fn::Sub:
                - mongodb+srv://${USERNAME}:${PASSWORD}@${MONGO_CONNECTION_STRING}
                - MONGO_CONNECTION_STRING:
                    Fn::Select:
                      - 1
                      - Fn::Split:
                          - mongodb+srv://
                          - Fn::GetAtt:
                              - MongoAtlasConstructMongoAtlasCluster48C24AB2
                              - ConnectionStrings.StandardSrv
                  USERNAME: core-env-test-root
                  PASSWORD:
                    Fn::Join:
                      - ""
                      - - "{{resolve:secretsmanager:"
                        - Ref: MongoAtlasConstructMongoAtlasDbUserSecretC02C1EF1
                        - :SecretString:password::}}
            - '"}'
    DependsOn:
      - MongoAtlasConstructMongoAtlasCluster48C24AB2
    UpdateReplacePolicy: Delete
    DeletionPolicy: Delete
    Metadata:
      aws:cdk:path: MongoClusterTestStack/MongoAtlasConstruct/MongoAtlasConnectionStringSecret/Resource
  MongoAtlasConstructMongoAtlasDbUserB8AF1243:
    Type: MongoDB::Atlas::DatabaseUser
    Properties:
      DatabaseName: admin
      Password: atlas-pwd
      ProjectId: 65c7a364ba5030155d686ffa
      Roles:
        - DatabaseName: admin
          RoleName: atlasAdmin
      Username: core-env-test-root
      Profile: core-env-test
    DependsOn:
      - MongoAtlasConstructMongoAtlasCluster48C24AB2
    UpdateReplacePolicy: Delete
    DeletionPolicy: Delete
    Metadata:
      aws:cdk:path: MongoClusterTestStack/MongoAtlasConstruct/MongoAtlasDbUser
  CDKMetadata:
    Type: AWS::CDK::Metadata
    Properties:
      Analytics: v2:deflate64:H4sIAAAAAAAA/12MMRKDIBBFz2IPGyHegDqNHiBDcHVW4+KwGAuHu2eMXar3XvG/hcM0d6grv4sO/azf9IKjyz7Myg3cosQtBTzdRe4pU+SiTGP+J27gh19X4lH5XZ6CIWGWxbMfMcHR/fq8uayUojj2CJPcPrYGY8FWkxDptHGmBaG9+AWXRPWBoQAAAA==
    Metadata:
      aws:cdk:path: MongoClusterTestStack/CDKMetadata/Default
    Condition: CDKMetadataAvailable
Conditions:
  CDKMetadataAvailable:
    Fn::Or:
      - Fn::Or:
          - Fn::Equals:
              - Ref: AWS::Region
              - af-south-1
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-east-1
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-northeast-1
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-northeast-2
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-south-1
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-southeast-1
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-southeast-2
          - Fn::Equals:
              - Ref: AWS::Region
              - ca-central-1
          - Fn::Equals:
              - Ref: AWS::Region
              - cn-north-1
          - Fn::Equals:
              - Ref: AWS::Region
              - cn-northwest-1
      - Fn::Or:
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-central-1
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-north-1
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-south-1
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-west-1
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-west-2
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-west-3
          - Fn::Equals:
              - Ref: AWS::Region
              - il-central-1
          - Fn::Equals:
              - Ref: AWS::Region
              - me-central-1
          - Fn::Equals:
              - Ref: AWS::Region
              - me-south-1
          - Fn::Equals:
              - Ref: AWS::Region
              - sa-east-1
      - Fn::Or:
          - Fn::Equals:
              - Ref: AWS::Region
              - us-east-1
          - Fn::Equals:
              - Ref: AWS::Region
              - us-east-2
          - Fn::Equals:
              - Ref: AWS::Region
              - us-west-1
          - Fn::Equals:
              - Ref: AWS::Region
              - us-west-2
Parameters:
  BootstrapVersion:
    Type: AWS::SSM::Parameter::Value<String>
    Default: /cdk-bootstrap/hnb659fds/version
    Description: Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip]
Rules:
  CheckBootstrapVersion:
    Assertions:
      - Assert:
          Fn::Not:
            - Fn::Contains:
                - - "1"
                  - "2"
                  - "3"
                  - "4"
                  - "5"
                - Ref: BootstrapVersion
        AssertDescription: CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI.

Code of Conduct

github-actions[bot] commented 3 weeks ago

Thanks for opening this issue! Please make sure to provide the following information to help us reproduce the issue:

Thanks for opening this issue. The ticket CLOUDP-252074 was created for internal tracking.

lockenj commented 3 weeks ago

After changing the name of my cluster to something with less than 21 characters and specifying and uncommenting // clusterName: this.#_clusterName, the above works.

However I've now discovered that the deployment will NOT finish if the database/collections are not created up front.

There are a lot of "scenarios" or specifics around these CDK resources that do NOT appear to reflected anywhere.

How can we get some of that knowledge captured to help future developers not burn energy/time exploring like I am doing at the moment?