search

aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.47k stars 3.83k forks source link

core: crossRegionReferences generates too large buildspec in FileAsset #30326

Open HansFalkenberg-Visma opened 3 months ago

HansFalkenberg-Visma commented 3 months ago

Describe the bug

Cross region references rely on using Lambda functions, which are deployed with the stack.

I am deploying to 17 regions with a mix of importing and exporting, so there will be 17 export writer functions and 17 readers. Per the 8 accounts I want to deploy to.

All cdk-asset deploys of the same Lambda is being done in a single FileAssetXx action. This fails because its build spec becomes too large.

The problem is somewhat lessened because -- I found -- cdk-asset will only be invoked once per environment (account:region) even if multiple stacks use it in that environment. This behavior is not obvious because a stack's ID is part of the command.

But it is still possible to exceed the limit even when deploying only to 3 accounts, if deploying to all 29 regions and having IDs that are not super short.

Expected Behavior

Cross region references in a pipeline are not limited by number of regions and accounts.

Current Behavior

InfrastructurePipeline/InfrastructurePipeline/Assets/FileAsset14 (InfrastructurePipelineAssetsFileAsset14268FCA88)
Failed to call UpdateProject, reason: Max buildspec length is 25600
(Service: AWSCodeBuild; Status Code: 400; Error Code: InvalidInputException; Request ID: ; Proxy: null)

Reproduction Steps

import { App, Stack, Stage } from 'aws-cdk-lib';
import {
  Effect,
  PolicyDocument,
  PolicyStatement,
  Role,
  ServicePrincipal,
} from 'aws-cdk-lib/aws-iam';
import { Bucket } from 'aws-cdk-lib/aws-s3';
import { CodePipeline, CodePipelineSource, ShellStep } from 'aws-cdk-lib/pipelines';

const app = new App();

const regions = [
 'af-south-1', 'ap-east-1', 'ap-northeast-1', 'ap-northeast-2', 'ap-northeast-3', 'ap-south-1', 'ap-south-2',
 'ap-southeast-1', 'ap-southeast-2', 'ap-southeast-3', 'ap-southeast-4', 'ca-central-1', 'ca-west-1',
 'eu-central-1', 'eu-central-2', 'eu-north-1', 'eu-south-1', 'eu-south-2', 'eu-west-1', 'eu-west-2', 'eu-west-3',
 'il-central-1', 'me-central-1', 'me-south-1', 'sa-east-1', 'us-east-1', 'us-east-2', 'us-west-1', 'us-west-2',
];

const accounts = ['123456789001', '123456789002', '123456789003', '123456789004'];

class MyPipeline extends Stack {
  constructor() {
    super(app, 'InfrastructurePipelineStack', {
      env: { account: '123456789099', region: 'eu-west-1' },
    });

    const pipeline = new CodePipeline(this, 'InfrastructurePipeline', {
      crossAccountKeys: true,
      synth: new ShellStep('Synth', {
        commands: [],
        input: CodePipelineSource.connection('o/r', 'b', { connectionArn: '' }),
      }),
    });

    for (const account of accounts) {
      const stage = new Stage(this, `Stage-${account}`);

      const exportingStack = new Stack(stage, `ExportingStack-${account}`, {
        env: { account: account, region: 'eu-west-1' },
        crossRegionReferences: true,
      });
      const exportedBucket = new Bucket(exportingStack, 'ExportedBucket');

      for (const region of regions) {
        const stack = new Stack(stage, `Stack-${account}-${region}`, {
          env: { account: account, region: region },
          crossRegionReferences: true,
        });
        new Role(stack, 'ImportingRole', {
          assumedBy: new ServicePrincipal('service.amazonaws.com'),
          inlinePolicies: {
            actions: new PolicyDocument({
              statements: [
                new PolicyStatement({
                  effect: Effect.ALLOW,
                  actions: ['s3:*'],
                  resources: [exportedBucket.bucketArn],
                }),
              ],
            }),
          },
        });
      }

      pipeline.addStage(stage);
    }
  }
}

new MyPipeline();

The resulting BuildSpec is about 32k. Even if only deploying to 3 accounts (a not uncommon pattern with ['test'], 'staging', 'production' and 'pipeline') it would be possible to exceed 25600 if identifiers were 20 characters longer than this example.

{
  "version": "0.2",
  "phases": {
    "install": {
      "commands": [
        "npm install -g cdk-assets@2"
      ]
    },
    "build": {
      "commands": [
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001afsouth110207D57.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-af-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001apeast15583CC4C.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-ap-east-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001apnortheast1327014FC.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-ap-northeast-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001apnortheast2DA4F6AF5.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-ap-northeast-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001apnortheast3F0C917D9.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-ap-northeast-3"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001apsouth18AF4448F.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-ap-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001apsouth2B3F8D462.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-ap-south-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001apsoutheast1228DED0C.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-ap-southeast-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001apsoutheast296BC5077.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-ap-southeast-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001apsoutheast3B108EDC8.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-ap-southeast-3"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001apsoutheast44FB59181.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-ap-southeast-4"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001cacentral1EB787D18.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-ca-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001cawest196843836.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-ca-west-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001eucentral12F0D5FB5.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-eu-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001eucentral23358FDCF.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-eu-central-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001eunorth10BE03AF1.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-eu-north-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001eusouth1A6A4B877.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-eu-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001eusouth2C3D35869.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-eu-south-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001euwest2A8E8BCAA.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-eu-west-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001euwest3B846D522.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-eu-west-3"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001ilcentral1B4F5BAA9.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-il-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001mecentral12995A8D5.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-me-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001mesouth1D1DDBAED.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-me-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001saeast1826DC9A5.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-sa-east-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001useast1B40BBABB.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-us-east-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001useast2A7D83BF3.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-us-east-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001uswest1D0A050EA.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-us-west-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001uswest2900A58F0.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-us-west-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002afsouth1A9A890AD.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-af-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002apeast1E2396BA9.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-ap-east-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002apnortheast1358FB29B.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-ap-northeast-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002apnortheast2E45F7D39.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-ap-northeast-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002apnortheast35E5D7B08.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-ap-northeast-3"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002apsouth184F84FC5.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-ap-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002apsouth242E1E678.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-ap-south-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002apsoutheast1868C395D.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-ap-southeast-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002apsoutheast2DB0112F0.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-ap-southeast-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002apsoutheast3FC7F7215.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-ap-southeast-3"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002apsoutheast4D1D52865.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-ap-southeast-4"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002cacentral12B53A514.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-ca-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002cawest1F96F5390.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-ca-west-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002eucentral1D6B1E887.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-eu-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002eucentral2DE4D33FA.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-eu-central-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002eunorth17B22D63D.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-eu-north-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002eusouth1F23BD9E7.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-eu-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002eusouth23ECE6AD5.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-eu-south-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002euwest2F1A6BF51.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-eu-west-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002euwest340F156C1.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-eu-west-3"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002ilcentral16876F965.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-il-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002mecentral10F5E9510.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-me-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002mesouth176B6F4A8.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-me-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002saeast1FAD8C4AF.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-sa-east-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002useast1536C8BA2.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-us-east-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002useast211EDE12B.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-us-east-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002uswest19E510473.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-us-west-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002uswest2BF50BFEC.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-us-west-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003afsouth111648FBA.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-af-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003apeast163F172C8.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-ap-east-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003apnortheast16047CF18.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-ap-northeast-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003apnortheast2F0E7F5A0.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-ap-northeast-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003apnortheast32ADB7D88.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-ap-northeast-3"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003apsouth10AAC2BF0.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-ap-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003apsouth2D70AEAC9.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-ap-south-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003apsoutheast16D0FC8F0.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-ap-southeast-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003apsoutheast2FB5FCC1F.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-ap-southeast-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003apsoutheast356FC751C.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-ap-southeast-3"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003apsoutheast4757670D0.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-ap-southeast-4"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003cacentral19B2A9A26.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-ca-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003cawest1E14E083B.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-ca-west-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003eucentral19F21F5D6.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-eu-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003eucentral289CB64A6.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-eu-central-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003eunorth134B0DAE1.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-eu-north-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003eusouth11F9E66BA.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-eu-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003eusouth23067775B.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-eu-south-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003euwest2D37CD1D1.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-eu-west-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003euwest350B35414.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-eu-west-3"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003ilcentral1C9E576C7.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-il-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003mecentral15743A928.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-me-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003mesouth1FAE1D361.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-me-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003saeast19C3B9507.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-sa-east-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003useast17B02A3D3.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-us-east-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003useast238196F94.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-us-east-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003uswest117F8A7A9.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-us-west-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003uswest23FC39B3E.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-us-west-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004afsouth1EE59763C.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-af-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004apeast191D1E59B.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-ap-east-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004apnortheast1DCEF24E6.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-ap-northeast-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004apnortheast27F26CFE3.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-ap-northeast-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004apnortheast349B5279D.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-ap-northeast-3"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004apsouth106E826A5.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-ap-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004apsouth2A772FBA1.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-ap-south-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004apsoutheast1F3D6CBDF.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-ap-southeast-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004apsoutheast2463FA3CD.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-ap-southeast-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004apsoutheast3C9BF8705.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-ap-southeast-3"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004apsoutheast4FC3AE12D.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-ap-southeast-4"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004cacentral150DAAD89.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-ca-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004cawest12423800D.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-ca-west-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004eucentral1D7A4CFE3.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-eu-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004eucentral250F06848.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-eu-central-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004eunorth1296091BA.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-eu-north-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004eusouth102FE31D3.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-eu-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004eusouth283DE98BA.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-eu-south-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004euwest2AB3726FD.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-eu-west-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004euwest3689D7C9F.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-eu-west-3"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004ilcentral1BE6000B7.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-il-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004mecentral1181220A1.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-me-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004mesouth11C1A8FC2.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-me-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004saeast11C91DA1A.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-sa-east-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004useast140DDCAB8.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-us-east-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004useast20C681DBC.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-us-east-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004uswest13582BC32.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-us-west-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004uswest2C7009EF7.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-us-west-2""
      ]
    }
  }
}

Possible Solution

Split into multiple FileAction actions if build spec becomes too large.

Additional Information/Context

No response

CDK CLI Version

2.143.0 (build 9f2bdf7)

Framework Version

No response

Node.js Version

v18.18.2

OS

Windows 10 Version 22H2

Language

TypeScript

Language Version

No response

Other information

No response

nindzyago commented 2 months ago

Faced same issue today. Any workarounds?

HansFalkenberg-Visma commented 2 months ago

I happened to be in the fortunate situation that is was possible to split deployment of a group of the accounts into a different pipeline. So even though I didn't really want a separate pipeline for this it wasn't a huge imposition either.

If that's not possible for you, I guess you could always manually manipulate the pipeline template and split the BuildSpec into two different actions as CDK should have done for you. If you want a starting point for that, I've done such manipulation for a totally different reason in https://github.com/aws/aws-cdk/issues/29592 and it has a .zip file with fully functional TypeScript code attached.

khushail commented 2 months ago

agree with your sggested workaround @HansFalkenberg-Visma . Marking the issue for further consideration by the team

cygnius commented 1 month ago

This is affecting us as well. We are deploying a basic hygiene stack to all our accounts which ensure compliance, kills zombie resources, support integration tests, etc. We are deploying to 27 regions and 7 accounts and facing the same problem. We are splitting the pipeline into multiple pipeline to bypass this issue for now. Requesting the team to prioritize this.

khushail commented 1 month ago

@cygnius, Since this has a workaround, it won't be possible to prioritise this and bypass other high priority issue for now. Please feel free to create an Internal SIM ticket through AWS Premium Support or TAMs ,if this issue needs IMMEDIATE prioritization from your side. Other way to get prioritisation on Github is through Upvotes. Users who are facing this issue, can upvote this issue and this should soon be considered by the team.