search

jorgebastida / awslogs

AWS CloudWatch logs for Humans™
Other
4.79k stars 333 forks source link

readme

awslogs

.. image:: https://badge.fury.io/py/awslogs.png :target: http://badge.fury.io/py/awslogs

.. image:: https://coveralls.io/repos/jorgebastida/awslogs/badge.svg :target: https://coveralls.io/r/jorgebastida/awslogs

awslogs is a simple command line tool for querying groups, streams and events from Amazon CloudWatch <http://aws.amazon.com/cloudwatch/>_ logs.

One of the most powerful features is to query events from several streams and consume them (ordered) in pseudo-realtime using your favourite tools such as grep::

$ awslogs get /var/log/syslog ip-10-1.* --start='2h ago' | grep ERROR

Features

Example

Running: awslogs get /var/logs/syslog ALL -s1d will return you events from any stream in the /var/logs/syslog group generated in the last day.

.. image:: https://github.com/jorgebastida/awslogs/raw/master/media/screenshot.png

Installation

You can easily install awslogs using pip::

$ pip install awslogs

If you are on OSX El Capitan, use the following (Why? Check Donald Stufft's comment here <https://github.com/pypa/pip/issues/3165#issuecomment-145856429>_) ::

$ pip install awslogs --ignore-installed six

You can also install it with brew <https://brew.sh/>_::

$ brew install awslogs

Options

Note: You need to provide to all these options a valid AWS region using --aws-region or AWS_REGION env variable.

Time options

While querying for logs you can filter events by --start -s and --end -e date.

Filter options

You can use --filter-pattern if you want to only retrieve logs which match one CloudWatch Logs Filter pattern. This is helpful if you know precisely what you are looking for, and don't want to download the entire stream.

For example, if you only want to download only the report events from a Lambda stream you can run::

$ awslogs get my_lambda_group --filter-pattern="[r=REPORT,...]"

Full documentation of how to write patterns: http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/FilterAndPatternSyntax.html

JSON logs

In a similar way than the aws-cli <http://docs.aws.amazon.com/cli/latest/userguide/controlling-output.html#controlling-output-filter>_ command, you can use --query to filter each of your json log lines and extract certain fields::

$ awslogs get my_lambda_group --query=message

This will only display the message field for each of the json log lines.

Using third-party endpoints

If you use tools like localstack, fakes3 or other, consider to change boto3 endpoint using --aws-endpoint-url or AWS_REGION env variable.

AWS IAM Permissions

The required permissions to run awslogs are contained within the CloudWatchLogsReadOnlyAccess <https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/iam-identity-based-access-control-cwl.html>_ AWS managed permissions. As of 2020-01-13, these are the permissions::

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "logs:Describe*",
                "logs:Get*",
                "logs:List*",
                "logs:StartQuery",
                "logs:StopQuery",
                "logs:TestMetricFilter",
                "logs:FilterLogEvents"
            ],
            "Effect": "Allow",
            "Resource": "*"
        }
    ]
}

Contribute

For more instructions see TESTING.md.

Helpful Links

How to provide AWS credentials to awslogs

Although, the most straightforward thing to do might be use --aws-access-key-id and --aws-secret-access-key, this will eventually become a pain in the ass.