data-dot-all / dataall

A modern data marketplace that makes collaboration among diverse users (like business, analysts and engineers) easier, increasing efficiency and agility in data projects on AWS.
https://data-dot-all.github.io/dataall/
Apache License 2.0
228 stars 82 forks source link

Synth step failure in deployment pipeline #784

Closed mvidhu closed 8 months ago

mvidhu commented 11 months ago

P.S. Don't attach files. Please, prefer add code snippets directly in the message body. HI Team,

While doing the recent upgrade, I moved cdk.json from github to SSM and also removed cdk.context.json file from github as it contains AWS account information which according to the github merge checks is a violation. Now my deployment pipeline is failing since tooling account does not have access to my deployment accounts (i.e. dev, test and prod) to read the vpc information. It is failing in Vpc.py file with error " Need to perform AWS calls for account "dev" but the current credentials are for "tooling" Where should we place cdkcontext.json file if we intend to use organization VPC set up and not let Data.all create vpc?

dlpzx commented 11 months ago

Hi @mvidhu,

Can you send the SSM parameter here (without the account and vpc ids)? It should have a very specific format that includes the context info inside the cdk.json.

Something like:

{
  "app": "python ./deploy/app.py",
  "context": {
    "@aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId": false,
    "@aws-cdk/aws-cloudfront:defaultSecurityPolicyTLSv1.2_2021": false,
    "@aws-cdk/aws-rds:lowercaseDbIdentifier": false,
    "@aws-cdk/core:stackRelativeExports": false,
    "availability-zones:account=111111111111:region=eu-west-1": [
       "eu-west-1a",
       "eu-west-1b",
       "eu-west-1c"
     ],
     "availability-zones:account=222222222222:region=eu-west-1": [
       "eu-west-1a",
       "eu-west-1b",
       "eu-west-1c"
     ],
    "tooling_region": "eu-west-1",
    "git_branch": "main",
    "repository_source": "github",
    "DeploymentEnvironments": [
        {
            "envname": "dev",
            "account": "222222222222",
            "region": "eu-west-1"
        }
    ]
  }
}
anmolsgandhi commented 10 months ago

Hello, it seems there hasn’t been any recent activity on this issue. Is there anything more we can do to assist, or shall we consider closing it if the problem has been resolved? Thanks