[Enhancement] auto-replace variables in scp/config file

[Brian969]

Describe the problem you want solved

• it's a lot of work to change the ASEA home region for new installs
• basic region adjustments require overriding the SCP's and maintaining through every upgrade
• will be hard to change the ASEA prefix (PBMM) in future

Describe the solution you'd like

• Variables used by customers in SCP's and config.json/yaml that are replaced automatically during SM execution (before raw generated):

  • ${HOME_REGION} - automatically resolves to the SM home/installation region (i.e. "ca-central-1") including quotes
  • ${GBL_REGION} - hardcode to "us-east-1" including quotes
  • ${ADDL_REGIONS_A}, ${ADDL_REGIONS_B}, ${ADDL_REGIONS_N} including quotes - lookup from config file per below
    • these variables can contain other variables, but not members of the ADDL_REGIONS array
  • ${ACCELERATOR_NAME} - should resolve to PBMM no quotes
  • ${ACCELERATOR_PREFIX} - should resolve to PBMMAccel- no quotes
  • ${ACCELERATOR_PREFIX_ND} - should resolve to PBMMAccel no quotes
  • ${ACCELERATOR_PREFIX_LND} - should resolve to pbmmaccel no quotes

• Add to config file:

  "replacements": {
    "addl_regions": {
    "a": ["${HOME_REGION}"],
    "b": ["${HOME_REGION}, ${GBL_REGION}"],
    "c": ["${HOME_REGION}, ${GBL_REGION}", "us-east-2", "us-west-1", "us-west-2"],
    "d": ["${HOME_REGION}", "${GBL_REGION}", "us-east-2", "us-west-1", "us-west-2", 
            "ap-southeast-2", "eu-central-1", "eu-north-1", "eu-west-1", "eu-west-2", "eu-west-3" ],
    "e": ["ap-northeast-1", "ap-northeast-2", "ap-south-1", "ap-southeast-1", "ap-southeast-2", 
            "${HOME_REGION}", "eu-central-1", "eu-north-1", "eu-west-1", "eu-west-2", "eu-west-3", 
            "sa-east-1", "${GBL_REGION}", "us-east-2", "us-west-1", "us-west-2" ]
  },
  "customer-variable": "customer value",
  }

  • while the config file definitions are all defined in lower case, the replacements MUST all be in Upper case:
  • addl_regions_a would replace all occurrences of ${ADDL_REGIONS_A}
  • ${CUSTOMER-VARIABLE} would use the value found in customer-variable, i.e. customer value

• An SCP file looking like this:

  "StringNotEquals": {
        "aws:RequestedRegion": [${ADDL_REGIONS_C}]
      },

  would change to:

  "StringNotEquals": {
        "aws:RequestedRegion": ["ca-central-1", "us-east-1", "us-east-2", "us-west-1", "us-west-2"]
      },

  This would also be executed against the config.json/yaml during /raw generation - for example all occurrences of ca-central-1 could be replaced with ${HOME_REGION}, etc.

Example - These lines in SCP's: "arn:aws:iam::*:role/${ACCELERATOR_PREFIX}*", "arn:aws:iam::*:role/${ACCELERATOR_NAME}Ops-*" "ec2:ResourceTag/Accel-P": "${ACCELERATOR_NAME}" "Resource": ["arn:aws:s3:::${ACCELERATOR_PREFIX_LND}-*", "arn:aws:s3:::cdktoolkit-*", "arn:aws:s3:::cf-templates-*"], "arn:aws:ssm:*:*:parameter/${ACCELERATOR_PREFIX_ND}*", "Resource": "arn:aws:iam::*:role/${ACCELERATOR_PREFIX}RDGW-Role", "Resource": "arn:aws:logs:::log-group:*${ACCELERATOR_PREFIX_ND}*", "Resource": ["arn:aws:firehose:*:*:deliverystream/${ACCELERATOR_PREFIX_ND}*", "arn:aws:kinesis:*:*:stream/${ACCELERATOR_PREFIX_ND}*"],

Would transform to these: "arn:aws:iam::*:role/PBMMAccel-*", "arn:aws:iam::*:role/PBMMOps-*" "ec2:ResourceTag/Accel-P": "PBMM" "Resource": ["arn:aws:s3:::pbmmaccel-*", "arn:aws:s3:::cdktoolkit-*", "arn:aws:s3:::cf-templates-*"], "arn:aws:ssm:*:*:parameter/PBMMAccel*", "Resource": "arn:aws:iam::*:role/PBMMAccel-RDGW-Role", "Resource": "arn:aws:logs:::log-group:*PBMMAccel*", "Resource": ["arn:aws:firehose:*:*:deliverystream/PBMMAccel*", "arn:aws:kinesis:*:*:stream/PBMMAccel*"],

(need to think about quote handling on regions - does it make sense? Should it be a string rather than an array to enable this?)

ADDITIONALLY

• remove all hardcoding of PBMM/PBMMAccel throughout codebase, except at the top of:
  • \src\installer\cdk\src\index.ts
  • \src\core\cdk\cdk.sh
• all other references should refer to these constants
  • references to PBMM in at least 9 other modules
• It should be easy to change the ASEA prefix (for new installs only) once these changes are made
• Move ACCELERATOR_NAME and ACCELERATOR_PREFIX to CFN Installer template customer supplied variables
  • Use PBMM and PBMMAccel- as default values
  • Add a check in the Code Pipeline to validate that these did not change since the original installation, if they did, immediately fail the Code Pipeline and stop install/upgrade

[Brian969]

For reference: '\${HOME_REGION}': region, '\${GBL_REGION}': GLOBAL_REGION, '\${ACCELERATOR_NAME}': acceleratorName, '\${ACCELERATOR_PREFIX}': acceleratorPrefix, '\${ACCELERATOR_PREFIX_ND}': accelPrefixNd, '\${ACCELERATOR_PREFIX_LND}': accelPrefixNd.toLowerCase(), '\${ORG_ADMIN_ROLE}': orgAdminRole!,

[Brian969]

#### PROBLEM1 - it failed to replace ${ORG_ADMIN_ROLE} in the SCP's - first run succeeds, second run fails due to SCP blocks. - weird - on re-examination, problem1 may not be an issue. #### PROBLEM 2 - double variables not properly replacing quotes ~~- "aws:RequestedRegion": "${ADDL_REGIONS_A}" replaced with: "aws:RequestedRegion": ["ca-central-1, us-east-1"] instead of: "aws:RequestedRegion": ["ca-central-1", "us-east-1"]~~ - and this in the _C case: "aws:RequestedRegion": ["ca-central-1, us-east-1", "us-east-2", "us-west-1", "us-west-2"] - missing two sets of quotes between ca-central-1 and us-east-1 - operator error, reformatted input file

QUESTION:

• Did you consider a) multi-part config files, and b) YAML config files when you implemented this? ANSWER: YES!