high cpu usage in in_cloudwatch

[tma-prismo]

Problem

high cpu usage between cloudwatch fetch ...

Steps to replicate

use top to check cpu, the ruby's cpu is close to 100% even when its not fetching from cloudwatch

Provide example config and message

Expected Behavior or What you need to ask

seems the "sleep 1" function in the "run" function in the in_cloudwatch_logs.rb need to be moved from the "if Time.now > @next_fetch_time" scope to the "until @finished" scope? otherwise the until loop is busy wait before the next fetch ...

Using Fluentd and CloudWatchLogs plugin versions

• OS version
• Bare Metal or within Docker or Kubernetes or others?
• Fluentd v0.12 or v0.14/v1.0
  • paste result of fluentd --version or td-agent --version
• Dependent gem versions
  • paste boot log of fluentd or td-agent
  • paste result of fluent-gem list, td-agent-gem list or your Gemfile.lock

[kenhys]

It can be reproducible during waiting next.

fluentd (1.12.0) fluent-plugin-cloudwatch-logs (0.13.3)

% cat fluent.in.conf
<system>
  log_level debug
</system>

<source>
  @type cloudwatch_logs
  region "#{ENV['AWS_REGION']}"
  aws_key_id "#{ENV['AWS_ACCESS_KEY_ID']}"
  aws_sec_key "#{ENV['AWS_SECRET_ACCESS_KEY']}"
  log_group_name test-group
  log_stream_name foobar
  tag cloud.in
</source>

<match>
  @type stdout
</match>

set proper log_group_name and log_stream_name on your environment.

[kenhys]

@tma-prismo

Thanks, I could send PR. #224

[cosmo0920]

I'd released @kenhys's patch as v0.13.4. Thanks for the report! :+1: