Adding Memory Monitoring Feature to EC2 Instance by Installing CloudWatch Agent

Hi,

I open this issue to give some instructions on how to enable memory usage monitoring on AWS EC2 instance. By default CPU & disk monitoring are enabled when an EC2 is started. With additional memory information, we can make better decision to resize the instance as needed and save cost.

Below are the simplified instructions for enabling memory monitoring by Installing CloudWatch Agent.

Step1 - Create IAM roles that enables the agent to collect metrics

AWS console -> IAM -> Roles -> Create Roles
Make sure that AWS service is selected under Select type of trusted entity.
For Choose a use case, choose EC2 under Common use cases,
Choose Next: Permissions.
Select the checkbox next to CloudWatchAgentServerPolicy. If necessary, use the search box to find the policy.
Choose Next: Review.
Confirm that CloudWatchAgentServerPolicy appears next to Policies. In the Role name, enter CloudWatchAgentServerRole. Then choose Create role.

Step 2 - Install CloudWatch Agent on EC2

SSH into EC2
For Centos (other OS requires different download link)
wget https://s3.amazonaws.com/amazoncloudwatch-agent/centos/amd64/latest/amazon-cloudwatch-agent.rpm
sudo rpm -U ./amazon-cloudwatch-agent.rpm

Step 3 - Create the Agent Configuration File

Download memMonitorConfig.json and upload to current directory in EC2

Step 4 - Attach IAM Role to a specific EC2

Open the Amazon EC2 console
In the navigation pane, choose Instances.
Select the instance, choose Actions, Security, Modify IAM role.
Select the CloudWatchAgentServerRole to attach to your instance, and choose Save.

Step 5 - Start the CloudWatch Agent

SSH into EC2
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file:memMonitorConfig.json
Verify CloudWatch Agent is running on EC2 sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a status

Step 6 - Check Memory Usage in AWS console

In AWS console -> CloudWatch -> Cutsom Namespaces -> CWAgent
Check mem_used_percent metric

When to resize the instance?

Suggestion from AWS doc:

Identify instances with a maximum CPU usage and memory usage of less than 40% over a four-week period. These are the instances that you will want to right size to reduce costs.

UBC-DSCI / jupyterhub-infrastructure

Adding Memory Monitoring Feature to EC2 Instance by Installing CloudWatch Agent #5