Closed rverma-jm closed 4 years ago
For some strange reason, the plugin keep getting shutdown after reading 1-2 events.
2020-04-30 04:50:36 +0000 [info]: using configuration file: <ROOT> <worker 0> <source> @type s3 @id cloudtrail @label @cloudtrail @log_level "debug" s3_bucket "jm-audit-logs" s3_region "ap-south-1" add_object_metadata false <web_identity_credentials> role_arn "arn:aws:iam::xx:role/jm-audit-fluent" role_session_name "jm-audit-s3-event-audit-cloudtrail" web_identity_token_file "/var/run/secrets/eks.amazonaws.com/serviceaccount/token" </web_identity_credentials> <sqs> queue_name "jm-audit-s3-event-audit-cloudtrail" </sqs> </source> <label @cloudtrail> <match input.s3> @type stdout @id firehose_ok </match> </label> </worker> <system> workers 1 root_dir "/var/log/fluentd" </system> </ROOT>
Fluentd logs
2020-04-30 04:50:37 +0000 [info]: #0 adding match in @cloudtrail pattern="input.s3" type="stdout" 2020-04-30 04:50:37 +0000 [info]: #0 adding source type="s3" 2020-04-30 04:50:37 +0000 [info]: #0 starting fluentd worker pid=16 ppid=6 worker=0 2020-04-30 04:50:37 +0000 [debug]: #0 [cloudtrail] Succeeded to create S3 client 2020-04-30 04:50:38 +0000 [debug]: #0 [cloudtrail] Succeeded to verify API keys 2020-04-30 04:50:38 +0000 [debug]: #0 [cloudtrail] Succeeded to create SQS client 2020-04-30 04:50:38 +0000 [debug]: #0 [cloudtrail] Succeeded to get SQS queue URL 2020-04-30 04:50:38 +0000 [info]: #0 fluentd worker is now running worker=0
Reading cloudtrail logs, this is a big json and nested json
2020-04-30 04:50:38 +0000 [debug]: #0 [cloudtrail] Records=[{"eventVersion"=>"2.1", "eventSource"=>"aws:s3", "awsRegion"=>"ap-south-1", "eventTime"=>"2020-04-30T04:50:22.568Z", "eventName"=>"ObjectCreated:Put", "userIdentity"=>{"principalId"=> .... } 2020-04-30 04:51:06 +0000 [info]: Received graceful stop 2020-04-30 04:51:07 +0000 [info]: #0 fluentd worker is now stopping worker=0 2020-04-30 04:51:07 +0000 [info]: #0 shutting down fluentd worker worker=0 2020-04-30 04:51:07 +0000 [info]: #0 shutting down input plugin type=:s3 plugin_id="cloudtrail" 2020-04-30 04:51:07 +0000 [info]: #0 shutting down output plugin type=:stdout plugin_id="firehose_ok" 2020-04-30 04:51:08 +0000 [warn]: #0 [cloudtrail] killing existing thread thread=#<Thread:0x00007f55afd92a40@in_s3@/usr/local/lib/ruby/gems/2.6.0/gems/fluentd-1.10.2/lib/fluent/plugin_helper/thread.rb:70 sleep> 2020-04-30 04:51:08 +0000 [warn]: #0 [cloudtrail] thread doesn't exit correctly (killed or other reason) plugin=Fluent::Plugin::S3Input title=:in_s3 thread=#<Thread:0x00007f55afd92a40@in_s3@/usr/local/lib/ruby/gems/2.6.0/gems/fluentd-1.10.2/lib/fluent/plugin_helper/thread.rb:70 aborting> error=nil
Couldn't figure out what is the issue, even in debug log level nothing is reflected.
@rverma-jm Hello!) how did you solve the error?
For some strange reason, the plugin keep getting shutdown after reading 1-2 events.
Fluentd logs
Reading cloudtrail logs, this is a big json and nested json
Couldn't figure out what is the issue, even in debug log level nothing is reflected.