awslabs / landing-zone-accelerator-on-aws

Deploy a multi-account cloud foundation to support highly-regulated workloads and complex compliance requirements.
https://aws.amazon.com/solutions/implementations/landing-zone-accelerator-on-aws/
Apache License 2.0
564 stars 450 forks source link

Pipeline times out at AuditManagerEnableOrganizationAdminAccount on eu-west-3 #416

Open zaid-themedcompany opened 8 months ago

zaid-themedcompany commented 8 months ago

Describe the bug When running LZA for healthcare configuration here and I deploy the solution to eu-west-3 (Paris) then the AWSAccelerator-Pipeline fails in the Organization stage and stack AWSAccelerator-OrganizationsStack-xxxxx-eu-west-3 runs till it times out, and fails to create logical Id AuditManagerEnableOrganizationAdminAccount

Is this happening config used for lza healthcare required Audit manager service which isn't available in Paris eu-west-3?

To Reproduce Run the lza with the healthcare configuration (excerpt below)

##################################################################################################################
# For additional configurable services, features, and property descriptions,                                     #
# please review our network configuration reference in our README.md:                                            #
# https://awslabs.github.io/landing-zone-accelerator-on-aws/classes/_aws_accelerator_config.SecurityConfig.html  #
##################################################################################################################

homeRegion: &HOME_REGION us-east-1
centralSecurityServices:
  ##################################################################################################################
  # Based upon AWS Security Reference Architecture (AWS SRA),                                                      #
  # Assigning delegated administrator to security tooling (Audit) account                                          #
  # https://docs.aws.amazon.com/prescriptive-guidance/latest/security-reference-architecture/security-tooling.html #
  ##################################################################################################################
  delegatedAdminAccount: Audit
  ebsDefaultVolumeEncryption:
    enable: true
    excludeRegions: []
  s3PublicAccessBlock:
    enable: true
    excludeAccounts: []
  scpRevertChangesConfig:
    enable: true
    snsTopicName: Security
  macie:
    enable: true
    excludeRegions: []
    policyFindingsPublishingFrequency: FIFTEEN_MINUTES
    publishSensitiveDataFindings: true
  guardduty:
    enable: true
    excludeRegions: []
    s3Protection:
      enable: true
      excludeRegions: []
    exportConfiguration:
      enable: true
      overrideExisting: true
      destinationType: S3
      exportFrequency: FIFTEEN_MINUTES
  auditManager:
    enable: true
    excludeRegions: []
    defaultReportsConfiguration:
      enable: true
      destinationType: S3
    lifecycleRules:
      - enabled: true
        abortIncompleteMultipartUpload: 7
        expiration: 1000
        noncurrentVersionExpiration: 1000
  detective:
    enable: false
    excludeRegions: []
  ##################################################################################################################
  # AWS Security Hub Configurations                                                                                #
  ##################################################################################################################
  securityHub:
    enable: true
    regionAggregation: true
    excludeRegions: []
    standards:

Expected behavior It should pass and the AWSAccelerator-Pipeline finishes successfully

Please complete the following information about the solution:

Screenshots

CloudFormation - error_enabling_audit_manager

Additional context

zaid-themedcompany commented 8 months ago

Also deleting the stack takes ages, I suppose again because there is no AWS Audit manager in eu-west-3? CloudFormation - deleting_failed_Stack

zaid-themedcompany commented 8 months ago

I tried a workaround to exclude Paris region from the audit Manager in security-config.yaml, but no luck:

  auditManager:
    enable: true
    excludeRegions:
      - eu-west-3

I also tried exclude Paris from the awsConfig using the following:

awsConfig:
  excludeRegions:
    - eu-west-3

But got the following error in the pipeline in Security_Resources stage and the stack AWSAccelerator-SecurityResourcesStack-xxxxx-eu-west-3

"Invalid request provided: NoAvailableConfigurationRecorder"

CloudFormation - Stack AWSAccelerator-SecurityResourcesStack-eu-west-3 2024-03-07 14-08-05

ye-yng commented 7 months ago

I tried a workaround to exclude Paris region from the audit Manager in security-config.yaml, but no luck:

  auditManager:
    enable: true
    excludeRegions:
      - eu-west-3

I also tried exclude Paris from the awsConfig using the following:

awsConfig:
  excludeRegions:
    - eu-west-3

But got the following error in the pipeline in Security_Resources stage and the stack AWSAccelerator-SecurityResourcesStack-xxxxx-eu-west-3

"Invalid request provided: NoAvailableConfigurationRecorder"

CloudFormation - Stack AWSAccelerator-SecurityResourcesStack-eu-west-3 2024-03-07 14-08-05

I can't say regarding the main issue, but we also ran into the NoAvailableConfigurationRecorder. This was mainly due to enabling a new region in Control Tower but not updating enrolled accounts. This leads to AWS Config in the new regions to not be initialized. The fix was to simply update enrolled accounts through Control Tower.

edududalan commented 3 months ago

Hello, any update here? @ye-yng how do you update de enrolled accounts to the new region? could you share de steps here?

gustavo-guerra-compasso commented 2 months ago

We are having the same problem with version 1.9.1 when enabling a secondary region.

This helped us but is not the expected behavior: https://github.com/awslabs/landing-zone-accelerator-on-aws/issues/517#issuecomment-2258839456

AWSAccelerator-SecurityResourcesStack-3111XXXXXXXX-us-west-2 |  0/89 | 9:38:02 PM | CREATE_FAILED        |
AWS::Config::ConfigRule               | AcceleratorEc2InstanceDetailedMonitoringEnabled
(AcceleratorEc2InstanceDetailedMonitoringEnabled9F032168) Resource handler returned message: "Invalid request provided:
NoAvailableConfigurationRecorder" (RequestToken: XXXXXXXX-dee3-ee84-d6cb-7d8268aXXXXX, HandlerErrorCode:
InvalidRequest)
ye-yng commented 2 months ago

Hello, any update here? @ye-yng how do you update de enrolled accounts to the new region? could you share de steps here?

Hi, I don't quite recall the details regarding our specific error, but updating enrolled accounts in Control Tower did solve it.

Assuming you enabled new regions in your Control Tower Landing Zone, perform the following steps to update enrolled accounts:

  1. Go to Control Tower > Organization
  2. If you enabled new regions in Control Tower you should see your accounts (other than the mandatory ones) with an Update Available Baseline State
  3. Update the accounts one by one

I couldn't find a more efficient solution at the time but this solved our issues.

adielLevyAllcloud commented 2 months ago

Have same issue but didn't add any new region v1.9.2

` AWSAccelerator-SecurityResourcesStack-557690590360-il-central-1 | 0/6 | 4:17:15 PM | CREATE_FAILED | AWS::Config::ConfigRule | ElbTlsHttpsListenersOnly (ElbTlsHttpsListenersOnlyB7785BFA) Resource handler returned message: "Invalid request provided: NoAvailableConfigurationRecorder" (RequestToken: c9e0670f-4bef-2789-8292-47f65faef6a6, HandlerErrorCode: InvalidRequest)

201 | new ManagedRule (/codebuild/output

`