fluent / fluent-plugin-s3

Amazon S3 input and output plugin for Fluentd
https://docs.fluentd.org/output/s3
314 stars 217 forks source link

Add debug log after s3 put #414

Open statefb opened 1 year ago

statefb commented 1 year ago

Is your feature request related to a problem? Please describe.

arch We'd like to send relatively large size data to Kinesis Data Streams. Due to the price problem, I'm considering send only meta data and body content is sent via S3. To do so, capturing debug log before putting in S3, parse it and then send to Kinesis is an idea. But if sending data to S3 takes some time, consumer may fail to fetch body content from S3 because placing in S3 possibly has not been completed and not existed. Can I add debug log after sending S3?

Describe the solution you'd like

Add log statement

Describe alternatives you've considered

@bucket.object(s3path).put(put_options)

+ log.debug "out_s3: completed put chunk to s3 #{dump_unique_id_hex(chunk.unique_id)} with metadata #{chunk.metadata} to s3://#{@s3_bucket}/#{s3path}"

Additional context

No response

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has been open 90 days with no activity. Remove stale label or comment or this issue will be closed in 30 days