"divided by 0" location="/opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/plugin/output.rb

[johndpknt]

Hey Team,

am tryng to create a conf to tail from a json file and put date to kinesis stream. Please find the below configurations.

<source>
  @type tail
  path /home/ubuntu/apps/src/wm-warehouse-backend/logs/data.json
  pos_file /var/log/td-agent/wm_data.pos
  <parse>
   @type json
   time_format "%Y-%m-%dT%H:%M:%S.%NZ"
   time_type string
  </parse>
 tag kinesis.wm.data
</source>

<match kinesis.wm.data.1>
        @type stdout
</match>

<match kinesis.wm.data>
  # plugin type
  @type kinesis_streams

  # your kinesis stream name
  stream_name wm-stream

  # AWS credentials
  aws_key_id XXXXxxxxxxx
  aws_sec_key xxxxxxxxxXXXX

  # AWS region
  region us-west-1

  # Use random value for the partition key
  random_partition_key true

  <buffer time>
    @type file
    path /home/ubuntu/apps/src/warehouse-backend/logs/data.buffer
    timekey 5
    # Frequency of ingestion
    flush_interval 1s
    # Parallelism of ingestion
    flush_thread_count
  </buffer>

But getting the error 2020-10-10 05:28:21 -0700 [warn]: #0 emit transaction failed: error_class=ZeroDivisionError error="divided by 0" location="/opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/plugin/output.rb:1289:in `%'" tag="kinesis.wm.data" 2020-10-10 05:28:21 -0700 [warn]: #0 suppressed same stacktrace

My json format looks like {"dateTime": 1602330699.7472975, "eventType": "adviewed", "gptAdPath": "/21751243814/WM_PUB2/WM_PUB2_StickyFooter", "gptId": "alc-sf", "isViewed": true, "packetId": "ckfyoo36o00003d74wp5zacmi", "siteId": 2, "viewCount": 4}

Please help.

[simukappu]

You did not set flush_thread_count parameter in your config. Set any value like this:

<match test>
  @type output_plugin
  <buffer ...>
    flush_thread_count 8
    # ...
  </buffer>
  # ...
</match>

See Performance Turning - Use flush_thread_count Parameter for more details.

In fluentd-1.11.2/lib/fluent/plugin/output.rb:1289, Fluentd devide the value by flush_thread_count. https://github.com/fluent/fluentd/blob/v1.11.2/lib/fluent/plugin/output.rb#L1289

This flush_thread_count seems to be zero because you didn't specify any value:

<buffer time>
    @type file
    # ...
    # Parallelism of ingestion
    flush_thread_count
  </buffer>

[simukappu]

Closing this issue for now. Please reopen if required.