Deploying CFT in Tel Aviv (il-central-1) fails

[mikeoleary]

Describe the bug

It appears that we cannot deploy the CFT in region il-central-1. Can we get support for this?

Expected behavior

CFT to deploy new stack will work.

Current behavior

Customer cannot deploy into Tel Aviv region (il-central-1). He gets the following error when trying the template for a Failover pair into a NEW VPC:

Resource handler returned message: "User: arn:aws:sts::xxxxxxxxxxxx:assumed-role/xxx/xxx@company.com is not authorized to perform: lambda:GetLayerVersion on resource: arn:aws:lambda:il-central-1:xxxxxxxxxxxx:layer:Klayers-p312-requests:3 because no resource-based policy allows the lambda:GetLayerVersion action (Service: Lambda, Status Code: 403, Request ID: xxx)" (RequestToken: xxx, HandlerErrorCode: AccessDenied)

Customer says he was able to get around this by creating a bucket specifically for himself and copying our templates into it. However, after that, he got a new error:

2024-07-01T12:08:56.958Z [9032]: error: AWS Cloud Client secret id arn:aws:secretsmanager:il-central-1:xxxxxxxxx:secret:xxxxxf5-bigIpSecret-xxxxxx is the wrong format

However, we double-checked this secret and it is in the correct format. (Secret itself is a string of letters and numbers without illegal special characters. IAM role allows permissions to secret).

Steps to reproduce

Deploy CFT into il-central-1

Note I cannot test in this region (il-central-f5) with my F5 account. I am unable to replicate customer's problem because of this.

[mikeshimkus]

Hi @mikeoleary, I created issue EC-526 for this, but cannot start work on it yet because our account also needs access to that region (it's been requested).

That said, for the first error you should have been able to get around it by providing an AMI for the bigIpCustomImageId parameter since looking up the AMI by name is the only thing GetLayerVersion is used for.

The second error is from runtime init. The validator for the secret ARN doesn't include any il regions and the one for simple secret name doesn't support uppercase. There are two options to work around this until we can fix:

1. Leave the secret ID parameter empty so that the template creates a new secret with a random value (see https://github.com/F5Networks/f5-aws-cloudformation-v2/blob/main/examples/failover/README.md#important-configuration-notes for info on that)
2. Recreate the existing secret, but make sure the name is all lowercase, e.g. xxxxxf5-bigipsecret-xxxxxx. Then use that non-ARN value for the bigIpSecretArn input. Note that it may be required to create an instance profile prior to deploying for this to work.

[mikeoleary]

@mikeshimkus, big thank you as always. I have asked customer to try this and also to subscribe to this issue.

[shiv-dasari]

bigIpCustomImageId: we used these AMIs--ami-084b3f263e7cff637 / ami-0e38f7892a301a8ca

Leave the secret ID parameter empty so that the template creates a new secret with a random value: yes we tried in call @mikeoleary

Then use that non-ARN value for the bigIpSecretArn input. : need assistance here

[mikeshimkus]

@shiv-dasari The options are either leave the secret ID empty or use non-ARN value for bigIpSecretArn (the latter assumes you have already created the secret).

We are going to release an update to F5 BIG-IP Runtime Init that should fix the issue with using the existing secret. I will post here when it's available.

[mikeshimkus]

Hi @shiv-dasari, for the secret ARN issue, please try updating the bigIpRuntimeInitPackageUrl parameter to https://cdn.f5.com/product/cloudsolutions/f5-bigip-runtime-init/v2.0.3/dist/f5-bigip-runtime-init-2.0.3-1.gz.run and then redeploy.

[shiv-dasari]

@mikeshimkus ---we are seeing this error- Resource handler returned message: "User: arn:aws:sts::068900102323:assumed-role/AWS_DCS_NetworkingAdmin/usa-shivshiva@deloitte.com is not authorized to perform: lambda:GetLayerVersion on resource: arn:aws:lambda:il-central-1:770693421928:layer:Klayers-p312-requests:3 because no resource-based policy allows the lambda:GetLayerVersion action (Service: Lambda, Status Code: 403, Request ID: db0fa450-3c54-43ca-9a82-98f196c0d2b3)" (RequestToken: 44adda57-3da7-95d2-1370-5239df6a8179, HandlerErrorCode: AccessDenied)

[shiv-dasari]

We don not have visibility/access to this account arn:aws:lambda:il-central-1:770693421928:layer:Klayers-p312-requests:3

[mikeshimkus]

For the GetLayerVersion error, you will either need to use the AMI ID for bigIpCustomImageId, or grant the required permissions to arn:aws:lambda:il-central-1:770693421928:layer:Klayers-p312-requests:3 in your IAM policy (the latter is only required when using the AMI lookup function, which should be bypassed if you provide bigIpCustomImageId.

[shiv-dasari]

@mikeshimkus --Thank you I have full access on my account

{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "", "Resource": "" } ]

AdministratorAccess | AWS managed - job function

I think it should be allowed on 770693421928: this account.

[shiv-dasari]

@mikeshimkus and @mikeoleary -- We were using this Image ID- in our last / even I checked again same error. bigIpCustomImageId | ami-0e38f7892a301a8ca

-

[mikeshimkus]

@shiv-dasari Looks like your IAM policy needs to allow lambda:GetLayerVersion

Example: https://github.com/F5Networks/f5-aws-cloudformation-v2/blob/ea11c96511a69435b7818ce3337709acc16e0230/examples/modules/access/access.yaml#L464

[shiv-dasari]

@mikeoleary --Need your assistance, can we connect on Monday. Please let me know.

[mikeoleary]

@shiv-dasari - ok will email you.

[shiv-dasari]

@mikeshimkusWe were able to add the Function stack to our S3 and run it, but we are still encountering this issue with the last stack (VM the building phase).

2024-07-16 20:03:18 UTC+0530 | ill-f5-stack-BigIpInstance02-1KORPCAEL3P6Q | CREATE_FAILED | The following resource(s) failed to create: [Bigip3NicInstance].

-- | -- | -- | --

2024-07-16 20:03:18 UTC+0530 | Bigip3NicInstance | CREATE_FAILED | Failed to receive 1 resource signal(s) within the specified duration

[shiv-dasari]

AWSTemplateFormatVersion: '2010-09-09' Description: >- This template creates BIG-IP PAYG or BYOL High Availability WAF solution. The template uses nested templates for provisioning network, access, and compute resources for hosting BIG-IP Failover solution. Metadata: AWS::CloudFormation::Interface: ParameterGroups:

• Label: default: Templates Location Parameters:
  • s3BucketName
  • s3BucketRegion
  • artifactLocation
• Label: default: Network Configuration Parameters:
  • numAzs
  • numSubnets
  • subnetMask
  • vpcCidr
  • bigIpMgmtAddress01
  • bigIpExternalSelfIp01
  • bigIpExternalVip01
  • bigIpInternalSelfIp01
  • bigIpMgmtAddress02
  • bigIpExternalSelfIp02
  • bigIpExternalVip02
  • bigIpInternalSelfIp02
  • bigIpPeerAddr
• Label: default: Amazon EC2 Configuration Parameters:
  • sshKey
  • bigIpInstanceProfile
• Label: default: DAG / Ingress Parameters:
  • provisionPublicIpMgmt
  • restrictedSrcAddressMgmt
  • restrictedSrcAddressApp
• Label: default: BIG-IP Configuration Parameters:
  • bigIpRuntimeInitPackageUrl
  • bigIpRuntimeInitConfig01
  • bigIpRuntimeInitConfig02
  • bigIpHostname01
  • bigIpHostname02
  • bigIpLicenseKey01
  • bigIpLicenseKey02
  • bigIpImage
  • bigIpCustomImageId
  • bigIpInstanceType
  • numNics
  • bigIpSecretArn
  • cfeS3Bucket
  • cfeVipTag
  • cfeTag
  • allowUsageAnalytics
• Label: default: Application Configuration Parameters:
  • appDockerImageName
  • provisionExampleApp
• Label: default: Resources Tags Parameters:
  • uniqueString
  • application
  • cost
  • environment
  • group
  • owner ParameterLabels: allowUsageAnalytics: default: Send anonymous statistics to F5 appDockerImageName: default: Application docker image name application: default: Application artifactLocation: default: Path to directory where the modules folder is located. ex. "examples/" bigIpCustomImageId: default: Custom Image Id bigIpHostname01: default: Hostname for BIG-IP instance 01 bigIpHostname02: default: Hostname for BIG-IP instance 02 bigIpImage: default: F5 BIG-IP Image bigIpInstanceType: default: Enter valid instance type. bigIpInstanceProfile: default: Instance profile bigIpLicenseKey01: default: License key for BIG-IP instance 01 bigIpLicenseKey02: default: License key for BIG-IP instance 02 bigIpPeerAddr: default: Static self IP address for peer device. bigIpRuntimeInitConfig01: default: BIG-IP Runtime Init config used for BIGIP instance A bigIpRuntimeInitConfig02: default: BIG-IP Runtime Init config used for BIGIP instance B bigIpRuntimeInitPackageUrl: default: Runtime Init Package cfeS3Bucket: default: CFE S3 bucket cfeTag: default: CFE Deployment tag value cfeVipTag: default: CFE VIP tag value cost: default: Cost Center environment: default: Environment bigIpExternalSelfIp01: default: External private ip address for BIGIP instance A bigIpExternalSelfIp02: default: External private ip address for BIGIP instance B bigIpExternalVip01: default: External secondary ip address for BIGIP instance A bigIpExternalVip02: default: External secondary ip address for BIGIP instance B group: default: Group bigIpInternalSelfIp01: default: Internal private ip address for BIGIP instance A bigIpInternalSelfIp02: default: Internal private ip address for BIGIP instance B bigIpMgmtAddress01: default: Management private ip address for BIGIP instance A bigIpMgmtAddress02: default: Management private ip address for BIGIP instance B bigIpSecretArn: default: ARN of Secrets Manager secret numAzs: default: Number of Availability Zones numNics: default: Interfaces numSubnets: default: Number of Subnets owner: default: Owner provisionExampleApp: default: Provision Example App provisionPublicIpMgmt: default: Provision Public IP addresses for the BIG-IP management interface restrictedSrcAddressApp: default: Restricted Source Address to Application restrictedSrcAddressMgmt: default: Restricted Source Address to BIG-IP s3BucketName: default: S3 Bucket where Templates are Located s3BucketRegion: default: S3 Bucket Region where Templates are Located sshKey: default: Supply the public key that will be used for SSH authentication to the BIG-IP and application virtual machines subnetMask: default: Subnet Mask uniqueString: default: Unique string vpcCidr: default: VPC CIDR Version: 3.5.0.0 Outputs: amiId: Condition: noCustomImageId Description: Ami lookup returned ami id. Value: !GetAtt [AmiInfo, Id] bastionHostInstanceId: Condition: noPublicIp Description: bastion instance id Value: !GetAtt [Bastion, Outputs.bastionInstanceId] bastionPublicIp: Condition: noPublicIp Description: bastion's public IP address Value: !GetAtt [Bastion, Outputs.bastionPublicIp] bigIpInstance01: Description: BIGIP instance A nested stack name Value: !GetAtt [BigIpInstance01, Outputs.stackName] bigIpInstanceMgmtPrivateIp01: Description: private management ip for BIGIP instance A Value: !GetAtt [BigIpInstance01, Outputs.bigIpManagementInterfacePrivateIp] bigIpManagementPublicIp01: Condition: usePublicIpMgmt Description: bigip A public management address. WARNING - For eval purposes only. Production should never have the management interface exposed to Internet Value: !GetAtt [Dag, Outputs.bigIpManagementEipAddress01] bigIpManagementSsh01: Condition: usePublicIpMgmt Description: ssh login to bigip A management address. WARNING - For eval purposes only. Production should never have the management interface exposed to Internet Value: !Join
• ''
• • 'ssh admin@'
    • !GetAtt [Dag, Outputs.bigIpManagementEipAddress01] bigIpManagement01Url443: Condition: usePublicIpMgmt Description: url to bigip A management address. WARNING - For eval purposes only. Production should never have the management interface exposed to Internet Value: !Join
• ''
• • 'https://'
    • !GetAtt [Dag, Outputs.bigIpManagementEipAddress01] bigIpInstance02: Description: BIGIP instance B nested stack name Value: !GetAtt [BigIpInstance02, Outputs.stackName] bigIpInstanceMgmtPrivateIp02: Description: private management ip for BIGIP instance B Value: !GetAtt [BigIpInstance02, Outputs.bigIpManagementInterfacePrivateIp] bigIpKeyPairName: Condition: createKeyPair Description: SSH key pair name Value: !GetAtt [Access, Outputs.keyPairName] bigIpManagementPublicIp02: Condition: usePublicIpMgmt Description: bigip B public management address. WARNING - For eval purposes only. Production should never have the management interface exposed to Internet Value: !GetAtt [Dag, Outputs.bigIpManagementEipAddress02] bigIpManagementSsh02: Condition: usePublicIpMgmt Description: ssh login to bigip B management address. WARNING - For eval purposes only. Production should never have the management interface exposed to Internet Value: !Join
• ''
• • 'ssh admin@'
    • !GetAtt [Dag, Outputs.bigIpManagementEipAddress02] bigIpManagement02Url443: Condition: usePublicIpMgmt Description: url to bigip B management address. WARNING - For eval purposes only. Production should never have the management interface exposed to Internet Value: !Join
• ''
• • 'https://'
    • !GetAtt [Dag, Outputs.bigIpManagementEipAddress02] bigIpSecretArn: Condition: createSecret Description: Secret ARN Value: !GetAtt [Access, Outputs.secretArn] cfeS3Bucket: Description: cfe s3 bucket created and used for cloud-failover-extension Value: !If
• useDefaultCfeS3Bucket
• !Join
  • ''
  • • !Ref 'uniqueString'
      • '-bigip-high-availability-solution'
• !Ref 'cfeS3Bucket' vipPublicUrl: Condition: usePublicIpVip Description: url to public vip address Value: !Join
• ''
• • 'https://'
    • !GetAtt [Dag, Outputs.bigIpExternalEipAddress03] Parameters: allowUsageAnalytics: AllowedValues:
• 'true'
• 'false' Default: 'true' Description: This deployment can send anonymous statistics to F5 to help us determine how to improve our solutions. If you select false statistics are not sent. Type: String appDockerImageName: Default: 'f5devcentral/f5-demo-app:latest' Description: Application docker image name Type: String application: Default: f5app Description: Application Tag. Type: String artifactLocation: AllowedPattern: ^.[0-9a-zA-Z]+/$ ConstraintDescription: key prefix can include numbers, lowercase letters, uppercase letters, hyphens (-), and forward slash (/). Default: f5-aws-cloudformation-v2/v3.5.0.0/examples/ Description: The path in the S3Bucket where the modules folder is located. Can include numbers, lowercase letters, uppercase letters, hyphens (-), and forward slash (/). Type: String bigIpCustomImageId: Default: '' Description: Provide BIG-IP AMI ID you wish to deploy. MaxLength: 255 Type: String bigIpHostname01: ConstraintDescription: Must be a valid hostname containing fewer than 63 characters. Default: 'gcsawsiscf501.local' Description: Supply the hostname you would like to use for the BIG-IP instance. The hostname must contain fewer than 63 characters. MaxLength: 63 Type: String bigIpHostname02: ConstraintDescription: Must be a valid hostname containing fewer than 63 characters. Default: 'gcsawsiscf502.local' Description: Supply the hostname you would like to use for the BIG-IP instance. The hostname must contain fewer than 63 characters. MaxLength: 63 Type: String bigIpImage: ConstraintDescription: Must be a valid F5 BIG-IP market place image Default: '16.1.4.2-0.0.3*BYOL-ALL Modules 2Boot' Description: F5 BIG-IP market place image Type: String bigIpInstanceProfile: Default: '' Description: Enter the name of an existing IAM instance profile with applied IAM policy to be associated to the BIG-IP virtual machine(s). Leave default to create a new instance profile. Type: String bigIpInstanceType: ConstraintDescription: Must be a valid EC2 instance type for BIG-IP Default: c5.large Description: Enter valid instance type. Type: String bigIpLicenseKey01: Default: '' Description: Supply the F5 BYOL license key for BIG-IP instance 01. Leave this parameter blank if deploying the PAYG solution. Type: String bigIpLicenseKey02: Default: '' Description: Supply the F5 BYOL license key for BIG-IP instance 02. Leave this parameter blank if deploying the PAYG solution. Type: String bigIpPeerAddr: Default: '10.203.146.20' Description: Provide the static address of the remote peer used for clustering. In this failover solution, clustering is initiated from the second instance (02) to the first instance (01) so you would provide the first instances Self IP address. Type: String bigIpRuntimeInitConfig01: Default: 'https://f5-cft-v2.s3.amazonaws.com/f5-aws-cloudformation-v2/v3.5.0.0/examples/failover/bigip-configurations/runtime-init-conf-3nic-byol-instance01-with-app.yaml' Description: 'REQUIRED - Supply a URL to the bigip-runtime-init configuration file in YAML or JSON format to use for f5-bigip-runtime-init configuration.' Type: String bigIpRuntimeInitConfig02: Default: 'https://f5-cft-v2.s3.amazonaws.com/f5-aws-cloudformation-v2/v3.5.0.0/examples/failover/bigip-configurations/runtime-init-conf-3nic-byol-instance02-with-app.yaml' Description: 'REQUIRED - Supply a URL to the bigip-runtime-init configuration file in YAML or JSON format to use for f5-bigip-runtime-init configuration.' Type: String bigIpRuntimeInitPackageUrl: Default: 'https://cdn.f5.com/product/cloudsolutions/f5-bigip-runtime-init/v2.0.2/dist/f5-bigip-runtime-init-2.0.2-1.gz.run' Description: URL for BIG-IP Runtime Init package. Type: String cfeS3Bucket: AllowedPattern: '^$|^(?=.{1,61}$)[0-9a-z]+([0-9a-z-.][0-9a-z])$' ConstraintDescription: 'S3 bucket name must be unique, can be between 3 and 63 characters long, and can contain only lower-case characters, numbers, periods, and dashes. It cannot contain underscores, end with a dash, have consecutive periods, or use dashes adjacent to periods.' Default: '' Description: 'Supply a unique name for a CFE S3 bucket created and used by Cloud Failover Extension.' Type: String cfeTag: Description: Cloud Failover deployment tag value. Type: String Default: bigip_high_availability_solution cfeVipTag: Description: Cloud Failover VIP tag value; provides private ip addresses to be assigned to VIP public ip. Type: String Default: '10.203.146.58,10.203.146.120' cost: Default: f5cost Description: Cost Center Tag. Type: String environment: Default: f5env Description: Environment Tag. Type: String bigIpExternalSelfIp01: AllowedPattern: '^$|^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5]).){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$' ConstraintDescription: IP address parameter must be in the form x.x.x.x Description: External Private IP Address for BIGIP instance A. Default: 10.203.146.53 Type: String bigIpExternalSelfIp02: AllowedPattern: '^$|^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5]).){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$' ConstraintDescription: IP address parameter must be in the form x.x.x.x Description: External Private IP Address for BIGIP instance B. Default: 10.203.146.117 Type: String bigIpExternalVip01: AllowedPattern: '^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5]).){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$' ConstraintDescription: IP address parameter must be in the form x.x.x.x Description: External Secondary Private IP Address for BIGIP instance A. Default: 10.203.146.58 Type: String bigIpExternalVip02: AllowedPattern: '^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5]).){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$' ConstraintDescription: IP address parameter must be in the form x.x.x.x Description: External Secondary Private IP Address for BIGIP instance B. Default: 10.203.146.120 Type: String group: Default: f5group Description: Group Tag. Type: String bigIpInternalSelfIp01: AllowedPattern: '^$|^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5]).){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$' ConstraintDescription: IP address parameter must be in the form x.x.x.x Description: Internal Private IP Address for BIGIP instance A. Default: 10.203.146.40 Type: String bigIpInternalSelfIp02: AllowedPattern: '^$|^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5]).){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$' ConstraintDescription: IP address parameter must be in the form x.x.x.x Description: Internal Private IP Address for BIGIP instance B. Default: 10.203.146.100 Type: String bigIpMgmtAddress01: AllowedPattern: '^$|^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5]).){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$' ConstraintDescription: IP address parameter must be in the form x.x.x.x Description: Management Private IP Address for BIGIP instance A. Default: 10.203.146.20 Type: String bigIpMgmtAddress02: AllowedPattern: '^$|^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5]).){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$' ConstraintDescription: IP address parameter must be in the form x.x.x.x Description: Management Private IP Address for BIGIP instance B. Default: 10.203.146.90 Type: String bigIpSecretArn: Default: '' Description: The ARN of an existing AWS Secrets Manager secret where the BIG-IP password used for clustering is stored. If left empty, a secret will be created. Type: String numAzs: Default: 2 Description: Number of Availability Zones to use in the VPC. Region must support number of availability zones entered. Min 1 Max 2. MaxValue: 2 MinValue: 1 Type: Number numNics: AllowedValues:
• 2
• 3 Default: 3 Description: Number of interfaces to create on BIG-IP instance. Maximum of 3 allowed. Minimum of 2 allowed. Type: Number numSubnets: Default: 4 Description: Indicate the number of subnets to create. A minimum of 4 subnets required when provisionExampleApp = false MaxValue: 8 MinValue: 2 Type: Number owner: Default: f5owner Description: Owner Tag. Type: String provisionExampleApp: AllowedValues:
• 'true'
• 'false' Default: 'true' Description: Flag to deploy the demo web application. Type: String provisionPublicIpMgmt: AllowedValues:
• 'true'
• 'false' Default: 'true' Description: Whether or not to provision public IP addresses for the BIG-IP management network interfaces. Type: String restrictedSrcAddressMgmt: AllowedPattern: '(\d{1,3}).(\d{1,3}).(\d{1,3}).(\d{1,3})/(\d{1,2})' ConstraintDescription: Must be a valid IP CIDR range of the form x.x.x.x/x. Description: REQUIRED - The IP address range used to SSH and access management GUI on the EC2 instances. MaxLength: '18' MinLength: '9' Type: String restrictedSrcAddressApp: AllowedPattern: '(\d{1,3}).(\d{1,3}).(\d{1,3}).(\d{1,3})/(\d{1,2})' ConstraintDescription: Must be a valid IP CIDR range of the form x.x.x.x/x. Description: REQUIRED - The IP address range that can be used to access web traffic (80/443) to the EC2 instances. MaxLength: '18' MinLength: '9' Type: String s3BucketName: AllowedPattern: ^[0-9a-zA-Z]+([0-9a-zA-Z-][0-9a-zA-Z])$ ConstraintDescription: >- S3 bucket name can include numbers, lowercase letters, uppercase letters, and hyphens (-). It cannot start or end with a hyphen (-). Default: f5-cft-v2 Description: >- REQUIRED - S3 bucket name for the modules. S3 bucket name can include numbers, lowercase letters, uppercase letters, and hyphens (-). It cannot start or end with a hyphen (-). Type: String s3BucketRegion: Default: us-east-1 Description: The AWS Region where the Quick Start S3 bucket (s3BucketName) is hosted. When using your own bucket, you must specify this value. Type: String sshKey: Default: '' Description: Supply the public key that will be used for SSH authentication to the BIG-IP, application, and bastion virtual machines. If left empty, one will be created. Type: String subnetMask: ConstraintDescription: 'Subnet mask must be in value of 16-28. Total number of subnets created from VPC must be greater than or equal to number of regions multiplied by number of subnets. Example: 4 AZ with 8 subnets requires VPC supernetting support 32 subnets.' Default: 28 Description: 'Mask for subnets. Valid values include 16-28. Note supernetting of VPC occurs based on mask provided; therefore, number of networks must be

  = to the number of subnets created. Mask for subnets. Valid values include 16-28.' MaxValue: 28 MinValue: 16 Type: Number uniqueString: ConstraintDescription: 'Must contain between 1 and 12 lowercase alphanumeric characters with first character as a letter.' AllowedPattern: ^[a-z][a-z0-9]{1,11}$ Description: Unique String used when creating object names or Tags. Type: String Default: gcsawsiscf5 vpcCidr: AllowedPattern: '^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5]).){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(1[6-9]|2[0-8]))$' ConstraintDescription: CIDR block parameter must be in the form x.x.x.x/16-28 Default: 10.203.146.0/24 Description: CIDR block for the VPC. Type: String Conditions: 2nic: !Or

  • !Condition '3nic'
  • !Equals

• 2
• !Ref 'numNics' 3nic: !Equals
  • 3
  • !Ref 'numNics' createBigIpInstanceProfile: !Equals
  • !Ref 'bigIpInstanceProfile'
  • "" createKeyPair: !Equals
  • ''
  • !Ref 'sshKey' createSecret: !Equals
  • ''
  • !Ref 'bigIpSecretArn' noCustomImageId: !Equals
  • ''
  • !Ref 'bigIpCustomImageId' noPublicIp: !Equals
  • 'false'
  • !Ref 'provisionPublicIpMgmt' sameAz: !Equals
  • '1'
  • !Ref 'numAzs' useDefaultCfeS3Bucket: !Equals
  • !Ref 'cfeS3Bucket'
  • '' usePublicIpMgmt: !Equals
  • 'true'
  • !Ref 'provisionPublicIpMgmt' usePublicIpVip: !Equals
  • 'true'
  • !Ref 'provisionExampleApp' Resources: Access: Type: 'AWS::CloudFormation::Stack' Properties: TemplateURL: !Sub
    • 'https://${S3Bucket}.s3.${S3Region}.amazonaws.com/${artifactLocation}modules/access/access.yaml'
    • S3Region: !Ref 's3BucketRegion' S3Bucket: !Ref 's3BucketName' Parameters: cfeTag: !Ref 'cfeTag' createAmiRole: 'true' createBigIpRoles: !If [createBigIpInstanceProfile, 'true', 'false'] createSecret: !If [createSecret, 'true', 'false'] createSshKey: !If [createKeyPair, 'true', 'false'] s3Bucket: !If
      • useDefaultCfeS3Bucket
      • !Join
      • ''
      • • !Ref 'uniqueString'
        • '-bigip-high-availability-solution'
      • !Ref 'cfeS3Bucket' secretArn: !If [createSecret, !Ref 'AWS::NoValue', !Ref 'bigIpSecretArn'] solutionType: failover uniqueString: !Ref 'uniqueString' application: !Ref 'application' cost: !Ref 'cost' environment: !Ref 'environment' group: !Ref 'group' owner: !Ref 'owner' AmiInfo: Type: 'Custom::AMIInfo' Condition: noCustomImageId Properties: OSName: !Ref 'bigIpImage' OwnerId: 'aws-marketplace' Region: !Ref 'AWS::Region' ServiceToken: !GetAtt [Function, Outputs.lambdaARN] Application: Type: 'AWS::CloudFormation::Stack' Condition: usePublicIpVip Properties: TemplateURL: https://application-s3-il.s3.il-central-1.amazonaws.com/application.yaml Parameters: appContainerName: !Ref 'appDockerImageName' applicationSubnet: !If
      • usePublicIpVip
      • !Join
      • ','
      • • !Select
        • '3'
        • !Split
          • ','
          • !GetAtt
          • Network
          • Outputs.subnetsA
      • !Join
      • ','
      • • !Select
        • '4'
        • !Split
          • ','
          • !GetAtt
          • Network
          • Outputs.subnetsA appSecurityGroupId: !GetAtt [Dag, Outputs.appSecurityGroupId] sshKey: !If [createKeyPair, !GetAtt [Access, Outputs.keyPairName], !Ref 'sshKey'] restrictedSrcAddress: !Ref 'restrictedSrcAddressApp' uniqueString: !Ref 'uniqueString' vpc: !GetAtt
      • Network
      • Outputs.vpcId application: !Ref 'application' cost: !Ref 'cost' environment: !Ref 'environment' group: !Ref 'group' owner: !Ref 'owner' Bastion: Type: 'AWS::CloudFormation::Stack' Condition: noPublicIp Properties: TemplateURL: !Sub
    • 'https://${S3Bucket}.s3.${S3Region}.amazonaws.com/${artifactLocation}modules/bastion/bastion.yaml'
    • S3Region: !Ref 's3BucketRegion' S3Bucket: !Ref 's3BucketName' Parameters: mgmtSubnet: !Join
      • ','
      • • !Select
        • '0'
        • !Split
        • ','
        • !GetAtt
          • Network
          • Outputs.subnetsA bastionSecurityGroupId: !GetAtt
      • Dag
      • Outputs.bastionSecurityGroupId sshKey: !If [createKeyPair, !GetAtt [Access, Outputs.keyPairName], !Ref 'sshKey'] restrictedSrcAddress: !Ref 'restrictedSrcAddressMgmt' uniqueString: !Ref 'uniqueString' vpc: !GetAtt
      • Network
      • Outputs.vpcId application: !Ref 'application' cost: !Ref 'cost' environment: !Ref 'environment' group: !Ref 'group' owner: !Ref 'owner' BigIpInstance01: Type: 'AWS::CloudFormation::Stack' Properties: TemplateURL: !Sub
    • 'https://${S3Bucket}.s3.${S3Region}.amazonaws.com/${artifactLocation}modules/bigip-standalone/bigip-standalone.yaml'
    • S3Region: !Ref 's3BucketRegion' S3Bucket: !Ref 's3BucketName' Parameters: allowUsageAnalytics: !Ref 'allowUsageAnalytics' bigIpPeerAddr: !Ref 'bigIpPeerAddr' bigIpPeerHostname: !Ref 'bigIpHostname02' bigIpRuntimeInitPackageUrl: !Ref 'bigIpRuntimeInitPackageUrl' bigIpRuntimeInitConfig: !Ref 'bigIpRuntimeInitConfig01' cfeS3Bucket: !If
      • useDefaultCfeS3Bucket
      • !Join
      • ''
      • • !Ref 'uniqueString'
        • '-bigip-high-availability-solution'
      • !Ref 'cfeS3Bucket' cfeTag: !Ref 'cfeTag' externalSelfPublicIpId: !GetAtt [Dag, Outputs.bigIpExternalEipAllocationId01] externalServicePublicIpIds: !If [usePublicIpVip, !GetAtt [Dag, Outputs.bigIpExternalEipAllocationId03], ''] externalSecurityGroupId: !GetAtt [Dag, Outputs.bigIpExternalSecurityGroup] externalSelfIp: !Ref 'bigIpExternalSelfIp01' externalServiceIps: !If [usePublicIpVip, !Ref 'bigIpExternalVip01', ''] externalSubnetId: !If
      • usePublicIpVip
      • !Join
      • ','
      • • !Select
        • '0'
        • !Split
          • ','
          • !GetAtt
          • Network
          • Outputs.subnetsA
      • !Join
      • ','
      • • !Select
        • '3'
        • !Split
          • ','
          • !GetAtt
          • Network
          • Outputs.subnetsA hostname: !Ref 'bigIpHostname01' imageId: !If
      • noCustomImageId
      • !GetAtt
      • AmiInfo
      • Id
      • !Ref 'bigIpCustomImageId' instanceIndex: '01' instanceProfile: !If
      • createBigIpInstanceProfile
      • !GetAtt
      • Access
      • Outputs.bigIpInstanceProfile
      • !Ref 'bigIpInstanceProfile' instanceType: !Ref 'bigIpInstanceType' internalSecurityGroupId: !If [3nic, !GetAtt [Dag, Outputs.bigIpInternalSecurityGroup], !Ref 'AWS::NoValue'] internalSelfIp: !If [3nic, !Ref 'bigIpInternalSelfIp01', !Ref 'AWS::NoValue'] internalSubnetId: !If
      • 3nic
      • !Join
      • ','
      • • !Select
        • '2'
        • !Split
          • ','
          • !GetAtt
          • Network
          • Outputs.subnetsA
      • !Ref 'AWS::NoValue' licenseKey: !Ref 'bigIpLicenseKey01' mgmtPublicIpId: !If [usePublicIpMgmt, !GetAtt [Dag, Outputs.bigIpManagementEipAllocationId01], ''] mgmtSecurityGroupId: !GetAtt [Dag, Outputs.bigIpMgmtSecurityGroup] mgmtAddress: !Ref 'bigIpMgmtAddress01' mgmtSubnetId: !Join
      • ','
      • • !Select
        • '1'
        • !Split
        • ','
        • !GetAtt
          • Network
          • Outputs.subnetsA numExternalPublicIpAddresses: !If [usePublicIpVip, 2, 1] numSecondaryPrivateIpAddresses: !If [usePublicIpVip, 1, 0] secretArn: !If [createSecret, !GetAtt [Access, Outputs.secretArn], !Ref 'bigIpSecretArn'] sshKey: !If [createKeyPair, !GetAtt [Access, Outputs.keyPairName], !Ref 'sshKey'] uniqueString: !Ref 'uniqueString' application: !Ref 'application' cost: !Ref 'cost' environment: !Ref 'environment' group: !Ref 'group' owner: !Ref 'owner' BigIpInstance02: Type: 'AWS::CloudFormation::Stack' Properties: TemplateURL: !Sub
    • 'https://${S3Bucket}.s3.${S3Region}.amazonaws.com/${artifactLocation}modules/bigip-standalone/bigip-standalone.yaml'
    • S3Region: !Ref 's3BucketRegion' S3Bucket: !Ref 's3BucketName' Parameters: allowUsageAnalytics: !Ref 'allowUsageAnalytics' bigIpPeerAddr: !Ref 'bigIpPeerAddr' bigIpPeerHostname: !Ref 'bigIpHostname01' bigIpRuntimeInitPackageUrl: !Ref 'bigIpRuntimeInitPackageUrl' bigIpRuntimeInitConfig: !Ref 'bigIpRuntimeInitConfig02' cfeTag: !Ref 'cfeTag' externalSelfPublicIpId: !GetAtt [Dag, Outputs.bigIpExternalEipAllocationId02] externalServicePublicIpIds: '' externalSecurityGroupId: !GetAtt [Dag, Outputs.bigIpExternalSecurityGroup] externalSelfIp: !Ref 'bigIpExternalSelfIp02' externalServiceIps: !If [usePublicIpVip, !Ref 'bigIpExternalVip02', ''] externalSubnetId: !If
      • usePublicIpVip
      • !Join
      • ','
      • • !Select
        • '0'
        • !Split
          • ','
          • !If [sameAz, !GetAtt [Network, Outputs.subnetsA], !GetAtt [Network, Outputs.subnetsB]]
      • !Join
      • ','
      • • !Select
        • '3'
        • !Split
          • ','
          • !If [sameAz, !GetAtt [Network, Outputs.subnetsA], !GetAtt [Network, Outputs.subnetsB]] hostname: !Ref 'bigIpHostname02' imageId: !If
      • noCustomImageId
      • !GetAtt
      • AmiInfo
      • Id
      • !Ref 'bigIpCustomImageId' instanceIndex: '02' instanceProfile: !If
      • createBigIpInstanceProfile
      • !GetAtt
      • Access
      • Outputs.bigIpInstanceProfile
      • !Ref 'bigIpInstanceProfile' instanceType: !Ref 'bigIpInstanceType' internalSecurityGroupId: !If [3nic, !GetAtt [Dag, Outputs.bigIpInternalSecurityGroup], !Ref 'AWS::NoValue'] internalSelfIp: !If [3nic, !Ref 'bigIpInternalSelfIp02', !Ref 'AWS::NoValue'] internalSubnetId: !If
      • 3nic
      • !Join
      • ','
      • • !Select
        • '2'
        • !Split
          • ','
          • !If [sameAz, !GetAtt [Network, Outputs.subnetsA], !GetAtt [Network, Outputs.subnetsB]]
      • !Ref 'AWS::NoValue' licenseKey: !Ref 'bigIpLicenseKey02' mgmtPublicIpId: !If [usePublicIpMgmt, !GetAtt [Dag, Outputs.bigIpManagementEipAllocationId02], ''] mgmtSecurityGroupId: !GetAtt [Dag, Outputs.bigIpMgmtSecurityGroup] mgmtAddress: !Ref 'bigIpMgmtAddress02' mgmtSubnetId: !Join
      • ','
      • • !Select
        • '1'
        • !Split
        • ','
        • !If [sameAz, !GetAtt [Network, Outputs.subnetsA], !GetAtt [Network, Outputs.subnetsB]] numExternalPublicIpAddresses: 1 numSecondaryPrivateIpAddresses: !If [usePublicIpVip, 1, 0] secretArn: !If [createSecret, !GetAtt [Access, Outputs.secretArn], !Ref 'bigIpSecretArn'] sshKey: !If [createKeyPair, !GetAtt [Access, Outputs.keyPairName], !Ref 'sshKey'] uniqueString: !Ref 'uniqueString' application: !Ref 'application' cost: !Ref 'cost' environment: !Ref 'environment' group: !Ref 'group' owner: !Ref 'owner' Dag: Type: AWS::CloudFormation::Stack Properties: TemplateURL: !Sub
    • https://${S3Bucket}.s3.${S3Region}.amazonaws.com/${artifactLocation}modules/dag/dag.yaml
    • S3Region: !Ref 's3BucketRegion' S3Bucket: !Ref 's3BucketName' Parameters: createAppSecurityGroup: true createFailoverIngress: true createInternalSecurityGroup: !If [3nic, 'true', 'false'] createExternalSecurityGroup: true createBastionSecurityGroup: !If [usePublicIpMgmt, false, true] cfeTag: !Ref 'cfeTag' cfeVipTag: !Ref 'cfeVipTag' environment: !Ref 'environment' externalSubnetAz1: !If
      • usePublicIpVip
      • !Join
      • ','
      • • !Select
        • '0'
        • !Split
          • ','
          • !GetAtt
          • Network
          • Outputs.subnetsA
      • !Join
      • ','
      • • !Select
        • '3'
        • !Split
          • ','
          • !GetAtt
          • Network
          • Outputs.subnetsA externalSubnetAz2: !If
      • usePublicIpVip
      • !Join
      • ','
      • • !Select
        • '0'
        • !Split
          • ','
          • !GetAtt
          • Network
          • Outputs.subnetsB
      • !Join
      • ','
      • • !Select
        • '3'
        • !Split
          • ','
          • !GetAtt
          • Network
          • Outputs.subnetsB group: !Ref 'group' internalSubnetAz1: !If
      • 3nic
      • !Select
      • '0'
      • !Split
        • ','
        • !GetAtt
        • Network
        • Outputs.subnetsA
      • !Ref 'AWS::NoValue' internalSubnetAz2: !If
      • 3nic
      • !Select
      • '0'
      • !Split
        • ','
        • !GetAtt
        • Network
        • Outputs.subnetsB
      • !Ref 'AWS::NoValue' numberPublicExternalIpAddresses: !If [usePublicIpVip, 3, 2] numberPublicMgmtIpAddresses: !If [usePublicIpMgmt, 2, 0] provisionExternalBigipLoadBalancer: false provisionInternalBigipLoadBalancer: false restrictedSrcAddressApp: !Ref 'restrictedSrcAddressApp' restrictedSrcAddressMgmt: !Ref 'restrictedSrcAddressMgmt' restrictedSrcPort: 443 uniqueString: !Ref 'uniqueString' vpc: !GetAtt
      • Network
      • Outputs.vpcId vpcCidr: !Ref 'vpcCidr' application: !Ref 'application' cost: !Ref 'cost' owner: !Ref 'owner' Function: Type: 'AWS::CloudFormation::Stack' Properties: TemplateURL: "https://s3.il-central-1.amazonaws.com/cf-templates-wxu2nbphx0p7-il-central-1/2024-05-22T124301.834Z2na-nestedfunction.yml" Parameters: amiLookupRole: !GetAtt
      • Access
      • Outputs.lambdaAmiExecutionRole createAmiLookupFunction: 'true' uniqueString: !Ref 'uniqueString' application: !Ref 'application' cost: !Ref 'cost' environment: !Ref 'environment' group: !Ref 'group' owner: !Ref 'owner' Network: Type: 'AWS::CloudFormation::Stack' Properties: TemplateURL: !Sub
    • 'https://${S3Bucket}.s3.${S3Region}.amazonaws.com/${artifactLocation}modules/network/network.yaml'
    • S3Region: !Ref 's3BucketRegion' S3Bucket: !Ref 's3BucketName' Parameters: numAzs: !Ref 'numAzs' numSubnets: !Ref 'numSubnets' setPublicSubnet1: !Ref 'provisionPublicIpMgmt' subnetMask: !Ref 'subnetMask' uniqueString: !Ref 'uniqueString' vpcCidr: !Ref 'vpcCidr' vpcTenancy: default application: !Ref 'application' cost: !Ref 'cost' environment: !Ref 'environment' group: !Ref 'group' owner: !Ref 'owner'

[shiv-dasari]

I am using this parent stack in AWS Israel region

And I added Function stack template into our S3 , then after we were able to ran with out any issue but still failing.

[mikeshimkus]

@shiv-dasari Do you have ssh access to the BIG-IP(s)? If yes can you share the content of /var/log/cloud/startup-script.log?

[shiv-dasari]

@mikeoleary --I have sent the requested below logs over email. var/log/cloud-init-output.log

/var/log/cloud/bigipruntimeinit.log /var/log/cloud/startup-script.log

/config/cloud/runtime-init.conf

[shiv-dasari]

@mikeshimkus , please find for the /var/log/cloud/startup-script.log [Uploading putty.log…]()