Open ryaninvents opened 6 years ago
curl https://s3.amazonaws.com/aws-cloudwatch/downloads/latest/awslogs-agent-setup.py -O
sudo python ./awslogs-agent-setup.py --non-interactive --region us-east-1
#
# ------------------------------------------
# CLOUDWATCH LOGS AGENT CONFIGURATION FILE
# ------------------------------------------
#
# --- DESCRIPTION ---
# This file is used by the CloudWatch Logs Agent to specify what log data to send to the service and how.
# You can modify this file at any time to add, remove or change configuration.
#
# NOTE: A running agent must be stopped and restarted for configuration changes to take effect.
#
# --- CLOUDWATCH LOGS DOCUMENTATION ---
# https://aws.amazon.com/documentation/cloudwatch/
#
# --- CLOUDWATCH LOGS CONSOLE ---
# https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#logs:
#
# --- AGENT COMMANDS ---
# To check or change the running status of the CloudWatch Logs Agent, use the following:
#
# To check running status: /etc/init.d/awslogs status
# To stop the agent: /etc/init.d/awslogs stop
# To start the agent: /etc/init.d/awslogs start
#
# --- AGENT LOG OUTPUT ---
# You can find logs for the agent in /var/log/awslogs.log
# You can find logs for the agent script in /var/log/awslogs-agent-setup.log
#
# ------------------------------------------
# CONFIGURATION DETAILS
# ------------------------------------------
# Refer to http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/AgentReference.html for details.
#
[general]
# Path to the CloudWatch Logs agent's state file. The agent uses this file to maintain
# client side state across its executions.
state_file = /var/awslogs/state/agent-state
#
## This queue size determine the max number of event batch loaded into memory. 10 is the default value.
## It can be reduced if the program consumes too much memory. 1 is the valid minimum value.
#queue_size = 10
#
## Each log file is defined in its own section. The section name doesn't
## matter as long as its unique within this file.
#[kern.log]
#
## Path of log file for the agent to monitor and upload.
#file = /var/log/kern.log
#
## Name of the destination log group.
#log_group_name = kern.log
#
## Name of the destination log stream. You may use {hostname} to use target machine's hostname.
#log_stream_name = {instance_id} # Defaults to ec2 instance id
#
## Format specifier for timestamp parsing. Here are some sample formats:
## Use '%b %d %H:%M:%S' for syslog (Apr 24 08:38:42)
## Use '%d/%b/%Y:%H:%M:%S' for apache log (10/Oct/2000:13:55:36)
## Use '%Y-%m-%d %H:%M:%S' for rails log (2008-09-08 11:52:54)
#datetime_format = %b %d %H:%M:%S # Specification details in the table below.
#
## A batch is buffered for buffer-duration amount of time or 32KB of log events.
## Defaults to 5000 ms and its minimum value is 5000 ms.
#buffer_duration = 5000
#
# Use 'end_of_file' to start reading from the end of the file.
# Use 'start_of_file' to start reading from the beginning of the file.
#initial_position = start_of_file
#
## Encoding of file
#encoding = utf-8 # Other supported encodings include: ascii, latin-1
#
#
#
# Following table documents the detailed datetime format specification:
# ----------------------------------------------------------------------------------------------------------------------
# Directive Meaning Example
# ----------------------------------------------------------------------------------------------------------------------
# %a Weekday as locale's abbreviated name. Sun, Mon, ..., Sat (en_US)
# ----------------------------------------------------------------------------------------------------------------------
# %A Weekday as locale's full name. Sunday, Monday, ..., Saturday (en_US)
# ----------------------------------------------------------------------------------------------------------------------
# %w Weekday as a decimal number, where 0 is Sunday and 6 is Saturday. 0, 1, ..., 6
# ----------------------------------------------------------------------------------------------------------------------
# %d Day of the month as a zero-padded decimal numbers. 01, 02, ..., 31
# ----------------------------------------------------------------------------------------------------------------------
# %b Month as locale's abbreviated name. Jan, Feb, ..., Dec (en_US)
# ----------------------------------------------------------------------------------------------------------------------
# %B Month as locale's full name. January, February, ..., December (en_US)
# ----------------------------------------------------------------------------------------------------------------------
# %m Month as a zero-padded decimal number. 01, 02, ..., 12
# ----------------------------------------------------------------------------------------------------------------------
# %y Year without century as a zero-padded decimal number. 00, 01, ..., 99
# ----------------------------------------------------------------------------------------------------------------------
# %Y Year with century as a decimal number. 1970, 1988, 2001, 2013
# ----------------------------------------------------------------------------------------------------------------------
# %H Hour (24-hour clock) as a zero-padded decimal number. 00, 01, ..., 23
# ----------------------------------------------------------------------------------------------------------------------
# %I Hour (12-hour clock) as a zero-padded decimal numbers. 01, 02, ..., 12
# ----------------------------------------------------------------------------------------------------------------------
# %p Locale's equivalent of either AM or PM. AM, PM (en_US)
# ----------------------------------------------------------------------------------------------------------------------
# %M Minute as a zero-padded decimal number. 00, 01, ..., 59
# ----------------------------------------------------------------------------------------------------------------------
# %S Second as a zero-padded decimal numbers. 00, 01, ..., 59
# ----------------------------------------------------------------------------------------------------------------------
# %f Microsecond as a decimal number, zero-padded on the left. 000000, 000001, ..., 999999
# ----------------------------------------------------------------------------------------------------------------------
# %z UTC offset in the form +HHMM or -HHMM (empty string if the the object is naive). (empty), +0000, -0400, +1030
# ----------------------------------------------------------------------------------------------------------------------
# %j Day of the year as a zero-padded decimal number. 001, 002, ..., 365
# ----------------------------------------------------------------------------------------------------------------------
# %U Week number of the year (Sunday as the first day of the week) as a zero padded 00, 01, ..., 53
# decimal number. All days in a new year preceding the first Sunday are considered
# to be in week 0.
# ----------------------------------------------------------------------------------------------------------------------
# %W Week number of the year (Monday as the first day of the week) as a decimal number. 00, 01, ..., 53
# All days in a new year preceding the first Monday are considered to be in week 0.
# ----------------------------------------------------------------------------------------------------------------------
# %c Locale's appropriate date and time representation. Tue Aug 16 21:30:00 1988 (en_US)
# ----------------------------------------------------------------------------------------------------------------------
This issue seems to hint that CloudWatch can pick up on JSON-formatted logs. I'll probably need to use JSON metric filters to get it working.
I'll probably need to set up logrotate.
Alternatively, I could skip setting up the Cloudwatch agent entirely, and use the stream
argument of the constructor to ship the logs to Cloudwatch.
EDIT: if I do this I might be interested in using something like awslogs-stream to implement it.
See instructions.