[Bug]: Error when trying to deploy a AtlasServerlessBasic construct

ludoblues commented 10 months ago

Is there an existing issue for this?

[X] I have searched the existing issues

CDK package version

v2.110.1

CFN Resource version

v2.1.0

CFN Resource Region

us-east-1

Current Behavior

I get this error when trying to use the Construct AtlasServerlessBasic: Resource handler returned message: "Unable to complete request: runtime error: invalid memory address or nil pointer dereference" (RequestToken: 1bfd9635-0a9e-bf60-9ef8-e7da011f65b0, HandlerErrorCode: GeneralServiceException)

I have just coppy/paste the serverless example avaible within this repo: https://github.com/mongodb/awscdk-resources-mongodbatlas/blob/main/examples/l3-resources/atlas-serverless-basic.ts

AWS CDK code to reproduce the issue

// This CDK L3 example creates a MongoDB Atlas project, cluster, databaseUser, and projectIpAccessList
import * as cdk from "aws-cdk-lib";
import { Construct } from "constructs";
import {
  AtlasServerlessBasic,
  ServerlessInstanceProviderSettingsProviderName,
} from "awscdk-resources-mongodbatlas";

interface AtlasStackProps {
  readonly orgId: string;
  readonly profile: string;
  readonly region: string;
  readonly ip: string;
  readonly continuousBackupEnabled: boolean;
  readonly terminationProtectionEnabled: boolean;
}

export class ToremoveStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const atlasProps = this.getContextProps();
    const atlasBasic = new AtlasServerlessBasic(this, "AtlasServerlessBasic", {
      serverlessProps: {
        projectId: atlasProps.orgId,
        profile: atlasProps.profile,
        continuousBackupEnabled: true,
        providerSettings: {
          providerName:
            ServerlessInstanceProviderSettingsProviderName.SERVERLESS,
        },
        terminationProtectionEnabled: true,
      },
      projectProps: {
        orgId: atlasProps.orgId,
      },

      ipAccessListProps: {
        accessList: [
          { ipAddress: atlasProps.ip, comment: "My first IP address" },
        ],
      },
      profile: atlasProps.profile,
    });
  }

  getContextProps(): AtlasStackProps {
    const orgId = this.node.tryGetContext("orgId");
    if (!orgId) {
      throw "No context value specified for orgId. Please specify via the cdk context.";
    }
    const profile = this.node.tryGetContext("profile") ?? "default";
    const terminationProtectionEnabled = this.node.tryGetContext(
      "terminationProtectionEnabled"
    );
    const continuousBackupEnabled = this.node.tryGetContext(
      "continuousBackupEnabled"
    );
    const region = this.node.tryGetContext("region") ?? "US_EAST_1";
    const ip = this.node.tryGetContext("ip");
    if (!ip) {
      throw "No context value specified for ip. Please specify via the cdk context.";
    }

    return {
      orgId,
      terminationProtectionEnabled,
      continuousBackupEnabled,
      profile,
      region,
      ip,
    };
  }
}

Steps To Reproduce

cdk deploy --context orgId="5aba5de1c0c6e3**" --context ip="37...*"

cdk synth

Resources:
  AtlasServerlessBasicprojectAtlasServerlessBasic2405B456:
    Type: MongoDB::Atlas::Project
    Properties:
      Name: atlas-project-4172791
      OrgId: 5aba5de1c0c6e3**********
      Profile: default
    Metadata:
      aws:cdk:path: ToremoveStack/AtlasServerlessBasic/project-AtlasServerlessBasic
  AtlasServerlessBasicserverlessAtlasServerlessBasic7186EADE:
    Type: MongoDB::Atlas::ServerlessInstance
    Properties:
      ContinuousBackupEnabled: true
      Name: atlas-serverless-2479174
      ProjectID: 5aba5de1c0c6e3**********
      ProviderSettings:
        ProviderName: SERVERLESS
      TerminationProtectionEnabled: true
      Profile: default
    DependsOn:
      - AtlasServerlessBasicprojectAtlasServerlessBasic2405B456
    Metadata:
      aws:cdk:path: ToremoveStack/AtlasServerlessBasic/serverless-AtlasServerlessBasic
  AtlasServerlessBasicdbuserAtlasServerlessBasic9A045D20:
    Type: MongoDB::Atlas::DatabaseUser
    Properties:
      DatabaseName: admin
      Password: atlas-pwd
      ProjectId:
        Fn::GetAtt:
          - AtlasServerlessBasicprojectAtlasServerlessBasic2405B456
          - Id
      Roles:
        - DatabaseName: admin
          RoleName: atlasAdmin
      Username: atlas-user
      Profile: default
    DependsOn:
      - AtlasServerlessBasicprojectAtlasServerlessBasic2405B456
    Metadata:
      aws:cdk:path: ToremoveStack/AtlasServerlessBasic/db-user-AtlasServerlessBasic
  AtlasServerlessBasicipaccesslistAtlasServerlessBasic601CE395:
    Type: MongoDB::Atlas::ProjectIpAccessList
    Properties:
      AccessList:
        - Comment: My first IP address
          IPAddress: 37.174.230.4
      ProjectId:
        Fn::GetAtt:
          - AtlasServerlessBasicprojectAtlasServerlessBasic2405B456
          - Id
      Profile: default
    DependsOn:
      - AtlasServerlessBasicprojectAtlasServerlessBasic2405B456
    Metadata:
      aws:cdk:path: ToremoveStack/AtlasServerlessBasic/ip-access-list-AtlasServerlessBasic
  CDKMetadata:
    Type: AWS::CDK::Metadata
    Properties:
      Analytics: v2:deflate64:H4sIAAAAAAAA/yXGuw2AIBAA0Fno4QQqezbACRAwOdAj4aOFcXdjrN7ToNQMkrmrCR+y2HGFe+nOZ242srGVUX38bgoF7Fjo4VRChNSmU0tQEiRLDVHUQR2PCPb3Bfs2JZJaAAAA
    Metadata:
      aws:cdk:path: ToremoveStack/CDKMetadata/Default
    Condition: CDKMetadataAvailable
Conditions:
  CDKMetadataAvailable:
    Fn::Or:
      - Fn::Or:
          - Fn::Equals:
              - Ref: AWS::Region
              - af-south-1
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-east-1
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-northeast-1
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-northeast-2
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-south-1
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-southeast-1
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-southeast-2
          - Fn::Equals:
              - Ref: AWS::Region
              - ca-central-1
          - Fn::Equals:
              - Ref: AWS::Region
              - cn-north-1
          - Fn::Equals:
              - Ref: AWS::Region
              - cn-northwest-1
      - Fn::Or:
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-central-1
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-north-1
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-south-1
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-west-1
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-west-2
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-west-3
          - Fn::Equals:
              - Ref: AWS::Region
              - il-central-1
          - Fn::Equals:
              - Ref: AWS::Region
              - me-central-1
          - Fn::Equals:
              - Ref: AWS::Region
              - me-south-1
          - Fn::Equals:
              - Ref: AWS::Region
              - sa-east-1
      - Fn::Or:
          - Fn::Equals:
              - Ref: AWS::Region
              - us-east-1
          - Fn::Equals:
              - Ref: AWS::Region
              - us-east-2
          - Fn::Equals:
              - Ref: AWS::Region
              - us-west-1
          - Fn::Equals:
              - Ref: AWS::Region
              - us-west-2
Parameters:
  BootstrapVersion:
    Type: AWS::SSM::Parameter::Value<String>
    Default: /cdk-bootstrap/hnb659fds/version
    Description: Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip]
Rules:
  CheckBootstrapVersion:
    Assertions:
      - Assert:
          Fn::Not:
            - Fn::Contains:
                - - "1"
                  - "2"
                  - "3"
                  - "4"
                  - "5"
                - Ref: BootstrapVersion
        AssertDescription: CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI.

Code of Conduct

[X] I agree to follow this project's Code of Conduct

github-actions[bot] commented 10 months ago

Thanks for opening this issue! Please make sure to provide the following information to help us reproduce the issue:

CDK code used by the customer
The output of cdk synth
List of Public CFN Resources that are activated in the AWS account with their release version
AWS Region where the CFN stack is running

Thanks for opening this issue. The ticket CLOUDP-221240 was created for internal tracking.

cmoyer2 commented 9 months ago

Running into the same issue.

Copied the AtlasServerlessBasic example verbatim and supplied the orgId and ip address. API key generated in Atlas works correctly and can make calls with it with the Atlas Admin API via Postman. API key is stored in AWS Secrets Manager in cfn/atlas/profile/default and the values are populated with Publickey and Privatekey. CloudFormation third party extensions are enabled.

From the error message, it looks like it's failing to create the Project, but not sure what I'm doing wrong here to fix the issue>

8:58:56 PM | CREATE_FAILED        | MongoDB::Atlas::Project             | AtlasServerlessBas...rlessBasic2405B456
Resource handler returned message: Unable to complete request: runtime error: invalid memory address or nil
 pointer dereference (RequestToken: xxxx-xxxxx-xxxx-xxxx-xxxxxxxxxxxxx, HandlerErrorCode: GeneralServiceException)

ludoblues commented 9 months ago

Thanks @cmoyer2 for reporting i am not the only one with this issue!

Just to mention, i also tried to ensure an already existing projectId, and i still have this issue by doing that.

lantoli commented 9 months ago

hi, thanks for creating the issue.

I'm unable to reproduce the same error, can you provide the full log in order to have more info?

Can you confirm that you have a Default profile created,

you're using the latest version? "awscdk-resources-mongodbatlas": "^3.1.0"

and CFN registered versions you're using for MongoDB::Atlas::ServerlessInstance, MongoDB::Atlas::DatabaseUser and MongoDB::Atlas::ProjectIpAccessList ?

thanks

ludoblues commented 9 months ago

Hi @lantoli

The full log is:

cdk deploy --context orgId="5aba5de1c0c6e3**********" --context ip="37.***.***.*"                                                                                                                                                                                                                           

✨  Synthesis time: 3.83s

TestCdkStack:  start: Building f736412a0b87dca4e63d502b1a66fd261c3f05beafd8b1e819ca3ee37b4742ee:current_account-current_region
TestCdkStack:  success: Built f736412a0b87dca4e63d502b1a66fd261c3f05beafd8b1e819ca3ee37b4742ee:current_account-current_region
TestCdkStack:  start: Publishing f736412a0b87dca4e63d502b1a66fd261c3f05beafd8b1e819ca3ee37b4742ee:current_account-current_region
TestCdkStack:  success: Published f736412a0b87dca4e63d502b1a66fd261c3f05beafd8b1e819ca3ee37b4742ee:current_account-current_region
TestCdkStack: deploying... [1/1]
TestCdkStack: creating CloudFormation changeset...
[··························································] (0/6)

11:33:40 PM | CREATE_FAILED        | MongoDB::Atlas::Project             | AtlasServerlessBas...lasServerlessBasic
Resource handler returned message: "Unable to complete request: runtime error: invalid memory address or nil pointer dereference" (RequestToken: 492722d0-dcdd-8b2c-bf86-2e95e1eac56c, HandlerErrorCode: GeneralServiceException)
11:33:40 PM | ROLLBACK_IN_PROGRESS | AWS::CloudFormation::Stack          | TestCdkStack
The following resource(s) failed to create: [AtlasServerlessBasicprojectAtlasServerlessBasic2405B456]. Rollback requested by user.
11:33:40 PM | ROLLBACK_IN_PROGRESS | AWS::CloudFormation::Stack          | TestCdkStack
The following resource(s) failed to create: [AtlasServerlessBasicprojectAtlasServerlessBasic2405B456]. Rollback requested by user.

My profile in secret manager is cfn/atlas/profile/deploysst so i named it that way in my Construct (i also tried to duplicate the secret in a cfn/atlas/profile/default name and it doesn't change anything):

new AtlasServerlessBasic(this, "AtlasServerlessBasic", {
      serverlessProps: {
        projectId: atlasProps.orgId,
        profile: "deploysst",
        continuousBackupEnabled: true,
        providerSettings: {
          providerName:
            ServerlessInstanceProviderSettingsProviderName.SERVERLESS,
        },
        terminationProtectionEnabled: true,
      },
      projectProps: {
        orgId: atlasProps.orgId,
      },

      ipAccessListProps: {
        accessList: [
          { ipAddress: atlasProps.ip, comment: "My first IP address" },
        ],
      },
      profile: "deploysst",
    });

I do am using the v3.1.0 version of "awscdk-resources-mongodbatlas" (i have just created a new project from scratch just before i met this error).

I am using the last version of the CFN ressources as well (v2.1.0 published the 27-10-2023 00:11:27 UTC+0200).

I can share more information with you in DM if needed to reproduce.

rooperas commented 9 months ago

I'm having the same issue.

I'm using our own serverless construct code that worked previously. The only difference between now and then seems to be that the AWS extensions (MongoDB::Atlas::ServerlessInstance etc) are using major version 1.x in the old service and 2.x now. Is there a way to downgrade the AWS extensions to version 1.x to test whether that would solve the issue?

lantoli commented 9 months ago

@rooperas I think AWS used to have a limitation there and doesn't allow to downgrade a CFN resource version. But you can try to unregister and register the other version in case you're allowed now.

do you mind to send me your log?

ludoblues commented 9 months ago

I just tried and i can't specify a version number when re-activating the extension. Plus i am not sure what would have been the conclusion if it worked anyway? It still needs to be fixed in v2.x.x no?

Zuhairahmed commented 9 months ago

@rooperas can you create a Support Ticket with AWS and send me number please? this way we can help to expediate from our side as well. you should be able to downgrade back to previous major versions, if you are not able to then this is likely an issue on the AWS CFN side.

lantoli commented 9 months ago

@ludoblues can you please remove this line and try again: projectId: atlasProps.orgId,

this line in the example is definitely wrong, although i'm still not able to reproduce the issue, it's working fine for me once this line is deleted.

ludoblues commented 9 months ago

Hi @lantoli ,

I have already tried, and i have just tried again, it fails the exact same way.

I have hardcoded all the values, and i also noticed multiple mistakes in the example (orgId passed instead of projectId, options in string instead of boolean in the TS interface). I don't see how this code could have been tested.

Are you using the v2.1.0 of the CF extensions too? Looks like it could be the key regarding the @rooperas comment

lantoli commented 9 months ago

@ludoblues I tried in us-west-2 and us-east-1, for instance in us-east-1 I used these CFN resource versions:

MongoDB::Atlas::DatabaseUser 2.1.0 MongoDB::Atlas::Project 2.1.0 MongoDB::Atlas::ProjectIpAccessList 2.0.0 MongoDB::Atlas::ServerlessInstance 2.0.0

Can you confirm that you're trying in region us_east_1 and what are the versions for the 4 CFN resources above.

Also can you confirm that the profile you're using has a key with sufficient permissions to create projects and serverless instances?

Sorry I can't reproduce your error, it's working fine for me.

ludoblues commented 9 months ago

Hi @lantoli

I tried in us-east-1 too.

I have just verified and i do try with the same CFN resources versions as you: MongoDB::Atlas::DatabaseUser 2.1.0 MongoDB::Atlas::Project 2.1.0 MongoDB::Atlas::ProjectIpAccessList 2.0.0 MongoDB::Atlas::ServerlessInstance 2.0.0

I have just checked the Organisation Permissions linked to my profile API key and here are the attached roles: "Organization Member, Organization Owner, Organization Project Creator, Organization Read Only"

Do you think something could be missing here?

Am i supposed to create an issue on the AWS support to get more information?

ludoblues commented 9 months ago

I got it!

I was able to get a much cleaner error message when trying to deploy with the CfnServerlessInstance construct idependently: "AccessDeniedException: User: arn:aws:sts::40**********:assumed-role/mongoDBAtlasCFNExecutionRole/73c108ea-d97c-89ce-441b-2ae4090d578b is not authorized to perform: secretsmanager:GetSecretValue on resource: cfn/atlas/profile/default because no identity-based policy allows the secretsmanager:GetSecretValue action

I just removed this part from the AtlasCDKExecution policy and it worked well:

 "Condition": {
  "StringEquals": {
    "aws:SourceAccount": "40**********"
  }
},

Which actually i don't get why since i am using a SSO login with the corresponding sso_account_id, is it really the SourceAccount when using SSO login? Looks like the most appropriate syntax could be to match the aws:username in a SSO login context, but it is another story so i close the issue.

Thanks for your help anyway!

github-actions[bot] commented 9 months ago

Thanks for opening this issue! Please make sure to provide the following information to help us reproduce the issue:

CDK code used by the customer
The output of cdk synth
List of Public CFN Resources that are activated in the AWS account with their release version
AWS Region where the CFN stack is running

Thanks for opening this issue. The ticket CLOUDP-222765 was created for internal tracking.

darksinge commented 9 months ago

I ran into this same issue while deploying with SST (which sits on top of CDK) and noticed OP was also using SST. I finally figured out the issue for me was that SST relies on cdk.context.json to cache values fetched from SSM, but if said file doesn't exist yet, SST uses dummy values during, or in preparation for the initial deployment. I was storing the API key-pair's ARN in SSM, so the ARN used to lookup the secret was something like dummy-value-for-/ssm/param/path instead of the actual secret's ARN.

Once I realized this, I deployed the stack without any Mongodb::Atlas::* resources, which populated cdk.context.json, then added back in the Mongodb::Atlas::* resources and redeployed, and the cryptic error disappeared.

Hope this helps someone so they don't have to waste hours troubleshooting like me 😅.

mongodb / awscdk-resources-mongodbatlas