search

fluent / fluentd-docs

This repository is deprecated. Go to fluentd-docs-gitbook repository.
49 stars 119 forks source link

The possibility that the setting value of buffer is incorrect #539

Closed 178inaba closed 6 years ago

178inaba commented 6 years ago

https://github.com/fluent/fluentd-docs/blob/4359d92db2a7c828ca244ae0893b0f5dc7a1f45b/docs/v1.0/apache-to-s3.txt#L77-L82

I tried 'Store Apache Logs into Amazon S3', but no file was created in S3.

The buffer tag section of the setting was modified as follows.

  <buffer time>
    @type file
    path /var/log/td-agent/s3
    timekey 60s
    timekey_wait 10s
    chunk_limit_size 256m
  </buffer>

When restarting fluentd, files are created in S3 every minute. I felt that the buffer setting described in "Store Apache Logs into Amazon S3" may be incorrect.

Reference URL: https://docs.fluentd.org/v1.0/articles/buffer-section#time

repeatedly commented 6 years ago

Document setting needs 1 hour for buffer flush. Did you wait 1 hour?

178inaba commented 6 years ago

@repeatedly Thank you for your reply.

I have not waited for an hour. Because I had set time_slice_format to %Y%m%d%H%M, I thought that the file would be sent to S3 every minute. https://github.com/fluent/fluentd-docs/blob/4359d92db2a7c828ca244ae0893b0f5dc7a1f45b/docs/v1.0/apache-to-s3.txt#L98

I recognized that waiting for an hour was right. Close the issue. Thank you!