search

aws-solutions / quota-monitor-for-aws

This solution leverages AWS Trusted Advisor and Service Quotas to monitor AWS resource usage and raise alerts.
Apache License 2.0
440 stars 116 forks source link

readme

Quota Monitor for AWS

🚀Solution Landing Page | 🚧Feature request | 🐛Bug Report | 📜Documentation Improvement

Note: For any relevant information outside the scope of this readme, please refer to the solution landing page and implementation guide.

Table of content

Solution overview

Quota Monitor for AWS is a reference implementation that provides a foundation for monitoring AWS services' quotas usage. Customers can leverage the solution to monitor quotas across services supported by Amazon Trusted Advisor and AWS Service Quotas; in multiple regions and multiple AWS accounts. The solution integrates with Amazon SNS and Slack to notify customers for service quotas approaching thresholds.

Architecture

The architecture can be broken down into different components, few which are installed in the monitoring account and others are installed in the monitored accounts. The monitoring account also known as hub account, collects all usage events from the monitored accounts (spokes) and raises notifications. Additionally, all the usage events are put on DynamoDB table in the hub account, which can be used to view historical trend of resource usage across all accounts.

Deployment scenarios:

The solution follows hub-spoke model and supports different deployment scenarios

hub: For the first two scenarios use quota-monitor-hub.template. For environments not using Organizations, use quota-monitor-hub-no-ou.template. Note: Hub template should be deployed in the monitoring account. For the first two scenarios, this account should also be your delegated administrator for StackSets in the organization.

spoke: Spoke templates are automatically deployed by StackSets for targeted Organizational Units. For hybrid environments and environments not using Organizations, deploy the spoke templates individually in the accounts where monitoring is needed. Note: ta-spoke.template should be deployed in us-east-1 ONLY. sq-spoke.template can be deployed in any region.

Installing pre-packaged solution template

Note: hub, hub-no-ou and sq-spoke templates can be deployed in ANY region; prerequisite and ta-spoke template can be deployed in us-east-1 ONLY.

Parameters for hub template

Note: Deployment Configuration parameter is not available in hub-no-ou template.

Parameters for spoke templates

Note: You may leave rest of the parameters to default.

Customization

The steps given below can be followed if you are looking to customize the solution or extend the solution with newer capabilities

Setup

Clone the repository and run the following commands to install dependencies

git clone https://github.com/aws-solutions/quota-monitor-for-aws.git
cd ./quota-monitor-for-aws
npm ci

(optional) Run the following commands to format and lint the project per the project standards

npm run prettier-format
npm run lint

Note: Following steps have been tested under above pre-requisites

Running unit tests for customization

Run unit tests to make sure added customization passes the tests.

cd ./deployment
chmod +x ./run-unit-tests.sh
./run-unit-tests.sh

✅ Ensure all unit tests pass. Review the generated coverage report.

Build

To build your customized distributable run build from the project root

npm run build:all

✅ All assets are now built.

Deploy

Run the following command from the root of the project

cd ./source/resources
npm ci

Bootstrap your CDK environment

npm run cdk -- bootstrap --profile <PROFILE_NAME>
npm run cdk -- deploy <STACK_NAME> --profile <PROFILE_NAME>
npm run orgHub:deploy -- deploy quota-monitor-hub --profile <PROFILE_NAME> (to deploy the hub template in org mode which works with stacksets)

Note:

✅ Solution stack is deployed with your customized code.

Independent spoke templates

There are three spoke templates packaged with the solution:

All three spoke templates (TA, SQ, and SNS) are independent standalone stacks that can be individually deployed. You can deploy these spoke stacks and route usage events and notifications to your preferred destinations.

For the SQ spoke stack, you can control which services to monitor by toggling the monitored status of the services in the DynamoDB table ServiceTable. The SNS spoke stack provides an additional option for routing notifications within spoke accounts. For deploying sq-spoke stack:

npm run cdk -- deploy quota-monitor-sq-spoke --parameters EventBusArn=<BUS_ARN> --profile <PROFILE_NAME>

Note: BUS_ARN, substitute the arn of the EventBridge bus where you want to send usage events

SSM Parameter Store based workflow

The solution provisions /QuotaMonitor/OUs and /QuotaMonitor/Accounts SSM Parameter Store. You can modify the parameters at any point after deployment to update the list of targeted organizational-units and accounts for monitoring.

File Structure

The project consists of several microservices, shared utility lambda layer and CDK resources

|-deployment/
  |-run-unit-test.sh              [ run all unit tests ]
  |-templates                     [ solution pre-baked templates ]
|-source/
  |-lambda
    |-services/
      |-cwPoller/                 [ microservice for polling CloudWatch metrics for quotas usage ]
      |-deploymentManager/        [ microservice for managing CloudFormation StackSet deployments ]
      |-helper/                   [ microservice for helper modules ]
      |-preReqManager/            [ microservice for fulfilling pre-requisites in the management account ]
      |-quotaListManager/         [ microservice for managing quota list that supports usage monitoring ]
      |-reporter/                 [ microservice for putting quota usage details on dynamodb ]
      |-slackNotifier/            [ microservice for raising alerts on slack ]
      |-snsPublisher/             [ microservice for publishing alerts to SNS ]
      |-taRefresher/              [ microservice for refreshing trusted advisor checks ]
    |-utilsLayer/                 [ lambda layer with shared modules, like logger, metrics, try/catch wrapper ]
  |-resources                     [ cdk resources to provision infrastructure ]
|-README.md
|-additional_files                [ CHANGELOG, CODE_OF_CONDUCT, LICENSE, NOTICE, sonar-project.properties etc.]

License

See license here

Collection of operational metrics

This solution collects anonymized operational metrics to help AWS improve the quality and features of the solution. For more information, including how to disable this capability, please see the implementation guide.

Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.

Licensed under the Apache License Version 2.0 (the "License"). You may not use this file except in compliance with the License. A copy of the License is located at

http://www.apache.org/licenses/LICENSE-2.0

or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or implied. See the License for the specific language governing permissions and limitations under the License.