search

aidan- / cloudtrailbeat

Easily export AWS CloudTrail events to ElasticSearch
MIT License
33 stars 8 forks source link

Multiple events #6

Closed idokaplan closed 8 years ago

idokaplan commented 8 years ago

Hi,

It looks like that there are a lot of "eventName" that cloudtrialbeat collects that are not available in CloudTrial AWS console (API activity history).

The list only includes API activity for create, modify, and delete API calls. I think that those events are not so important to collect.

Do you have an idea how to collect only "eventName" that are listed in the API activity history?

Thanks! Ido

aidan- commented 8 years ago

Interesting requirement. As it's currently configured, the easiest way to filter out specific events (or services) would be to send the output of cloudtrailbeat to a Logstash instance that is configured with a filter to drop messages with certain eventName values/prefixes. This would give you the freedom to filter or alter the data as you wish.

Off the top of my head, something like this could work for the filter: (this hasn't been tested):

filter {
    grok {
        match => { "message" => "^(Describe|Get)" }
        add_field => { "matched" }
    }
    if [matched] {
        drop { }
    }
}
idokaplan commented 8 years ago

I understand, thanks! I guess that your solution will work, but I didn't test it.

I have decided to forwards all logs for now, I would like to see how much messages I will get (and will filter via Kibana).

Thanks again. Ido