lukewaite / logstash-input-cloudwatch-logs

Input plugin for Logstash to stream events from CloudWatch Logs
Other
142 stars 74 forks source link

Multiple entries for for log_group #13

Closed kishore-vkrn closed 7 years ago

kishore-vkrn commented 7 years ago

Hi,

We have the following loggroup patterns /aws/lambda/app-name Can we have input log_group => ["logname1", "logname2"] does not work

Logstash version: 5.1

lukewaite commented 7 years ago

Hi @kishorramanan - I've just merged in #9 which allows you to use a prefix for importing multiple log groups.

Does that meet your need?

KIVagant commented 7 years ago

@lukewaite , error with config:

input {
   cloudwatch_logs {
     access_key_id => "123"
     secret_access_key => "456"
     region => "us-east-1"
     log_group_prefix => "myprefix"
   }
 }

Log output:

[2017-06-06T14:10:00,250][ERROR][logstash.inputs.cloudwatch_logs] Missing a required setting for the cloudwatch_logs input plugin:

  input {
    cloudwatch_logs {
      log_group => # SETTING MISSING
      ...
    }
  }

And, it's still better to be able to set array of logs groups instead of prefix.

lukewaite commented 7 years ago

Hi there.

'log_group_prefix' is a Boolean, and instructs whether to treat the required 'log_group' setting as an absolute or prefix. On Tue, Jun 6, 2017 at 10:14 AM Eugene Glotov notifications@github.com wrote:

@lukewaite https://github.com/lukewaite , error with config:

input { cloudwatch_logs { access_key_id => "123" secret_access_key => "456" region => "us-east-1" log_group_prefix => "myprefix" } }

Log output:

[2017-06-06T14:10:00,250][ERROR][logstash.inputs.cloudwatch_logs] Missing a required setting for the cloudwatch_logs input plugin:

input { cloudwatch_logs { log_group => # SETTING MISSING ... } }

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lukewaite/logstash-input-cloudwatch-logs/issues/13#issuecomment-306499290, or mute the thread https://github.com/notifications/unsubscribe-auth/AAlukuROWu_sXZT-1zvRi_FqXvCBSilIks5sBV7agaJpZM4LVw9f .

KIVagant commented 7 years ago

Ok, thank you.

lukewaite commented 7 years ago

Sorry for the short response, and lack of formatting. Am travelling and only had access to email.

Did that work for you?

KIVagant commented 7 years ago

I didn't have a time to test this yet.

lukewaite commented 7 years ago

And, it's still better to be able to set array of logs groups instead of prefix.

I'd be happy to accept a PR introducing this behaviour.

I'm planning a rewrite, for a v1 release, and can likely include this. I can't offer a timeline yet, though.

lukewaite commented 7 years ago

@kishorramanan I've just tagged v1.0.0.pre, a pre-release which adds support for specifying an array of log_groups.

https://github.com/lukewaite/logstash-input-cloudwatch-logs/releases/tag/v1.0.0.pre