search

logstash-plugins / logstash-input-salesforce

Logstash input for pulling objects from salesforce
Apache License 2.0
16 stars 29 forks source link

Salesforce Pipeline aborted due to error in logstash #23

Open nareshahi opened 3 years ago

nareshahi commented 3 years ago

Hi All, Please help if any one successfully integrated Salesforce cloud logs in SIEM. I am using the following config pipeline file in logstash to fetch the logs from cloud. but Pipeline aborted due to following error.

Kindly help me how i can overcome this issue.

input { salesforce { client_id => "yyyyyy" client_secret => "Aaaaaaaa" username => "" password => "" security_token => "" sfdc_object_name => "uuuuuu" } }

filter { mutate { rename => ["host", "host.ip"] } }

output {

stdout { codec => rubydebug { metadata => true } }

lumberjack { hosts => ["xxxxxx"] codec => json ssl_certificate => "/etc/logstash/certs/logstash.cert" port => xxxx } }

Error

[2020-10-14T10:36:59,350][ERROR][logstash.agent ] Failed to execute action {:id=>:salesforce, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create, action_result: false", :backtrace=>nil} [2020-10-14T10:37:11,399][INFO ][logstash.javapipeline ] Starting pipeline {:pipeline_id=>"salesforce", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>125, :thread=>"#"} [2020-10-14T10:38:12,339][ERROR][logstash.javapipeline ] Pipeline aborted due to error {:pipeline_id=>"salesforce", :exception=>#, :backtrace=>["org/jruby/ext/socket/RubyTCPSocket.java:119:in initialize'", "org/jruby/RubyIO.java:1155:inopen'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:941:in block in connect'", "org/jruby/ext/timeout/Timeout.java:99:intimeout'", "org/jruby/ext/timeout/Timeout.java:75:in timeout'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:939:inconnect'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:924:in do_start'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:913:instart'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:1465:in request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/adapter/net_http.rb:82:inperform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/adapter/net_http.rb:40:in block in call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/adapter/net_http.rb:87:inwith_net_http_connection'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/adapter/net_http.rb:32:in call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday_middleware-0.14.0/lib/faraday_middleware/response_middleware.rb:31:incall'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/middleware/mashify.rb:8:in call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/request/url_encoded.rb:15:incall'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/rack_builder.rb:139:in build_response'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/connection.rb:377:inrun_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/connection.rb:177:in post'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/middleware/authentication.rb:24:inauthenticate!'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/middleware/authentication.rb:18:in call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday_middleware-0.14.0/lib/faraday_middleware/request/encode_json.rb:24:incall'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/middleware/multipart.rb:16:in call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/middleware/mashify.rb:8:incall'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/rack_builder.rb:139:in build_response'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/connection.rb:377:inrun_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/connection.rb:140:in get'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/concerns/verbs.rb:37:inblock in define_verb'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/concerns/verbs.rb:63:in block in define_api_verb'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/concerns/api.rb:128:indescribe'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-salesforce-3.0.6/lib/logstash/inputs/salesforce.rb:90:in register'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:191:inblock in register_plugins'", "org/jruby/RubyArray.java:1792:in each'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:190:inregister_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:280:in start_inputs'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:244:instart_workers'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:145:in run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:104:inblock in start'"], :thread=>"#"} [2020-10-14T10:38:12,355][ERROR][logstash.agent ] Failed to execute action {:id=>:salesforce, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create, action_result: false", :backtrace=>nil} [2020-10-14T10:38:14,459][INFO ][logstash.javapipeline ] Starting pipeline {:pipeline_id=>"salesforce", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>125, :thread=>"#"}

nareshahi commented 3 years ago

Hello Folks, please help if anyone face this issue.

Madere commented 3 years ago

Hi nareshahi,

you an authentication failure (as you already may have figured out). This might be due to the missing salesforce input field: "api_version". Add it to your salesforce input config and see if that helps. In my case it is: api_version => "43.0" Regards, Martin.

nareshahi commented 3 years ago

Thanks Martin. Tried with your provided input but it is still same suition.

[2020-10-20T10:47:08,602][ERROR][logstash.javapipeline ] Pipeline aborted due to error {:pipeline_id=>"salesforce", :exception=>#, :backtrace=>["org/jruby/ext/socket/RubyTCPSocket.java:119:in initialize'", "org/jruby/RubyIO.java:1155:inopen'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:941:in block in connect'", "org/jruby/ext/timeout/Timeout.java:99:intimeout'", "org/jruby/ext/timeout/Timeout.java:75:in timeout'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:939:inconnect'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:924:in do_start'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:913:instart'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:1465:in request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/adapter/net_http.rb:82:inperform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/adapter/net_http.rb:40:in block in call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/adapter/net_http.rb:87:inwith_net_http_connection'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/adapter/net_http.rb:32:in call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday_middleware-0.14.0/lib/faraday_middleware/response_middleware.rb:31:incall'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/middleware/mashify.rb:8:in call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/request/url_encoded.rb:15:incall'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/rack_builder.rb:139:in build_response'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/connection.rb:377:inrun_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/connection.rb:177:in post'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/middleware/authentication.rb:24:inauthenticate!'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/middleware/authentication.rb:18:in call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday_middleware-0.14.0/lib/faraday_middleware/request/encode_json.rb:24:incall'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/middleware/multipart.rb:16:in call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/middleware/mashify.rb:8:incall'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/rack_builder.rb:139:in build_response'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/connection.rb:377:inrun_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/connection.rb:140:in get'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/concerns/verbs.rb:37:inblock in define_verb'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/concerns/verbs.rb:63:in block in define_api_verb'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/concerns/api.rb:128:indescribe'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-salesforce-3.0.6/lib/logstash/inputs/salesforce.rb:90:in register'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:191:inblock in register_plugins'", "org/jruby/RubyArray.java:1792:in each'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:190:inregister_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:280:in start_inputs'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:244:instart_workers'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:145:in run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:104:inblock in start'"], :thread=>"#"}

nareshahi commented 3 years ago

Hello Martin, you mentioned in my case, can you please provide you Salesforce.conf so that i would map if anything missing.

regards nareshahi

nareshahi commented 3 years ago

Hi Martin, i am have proxy to reach salesforce cloud. please suggest what setting i need to do on .conf pipeline to reach slaesforce via proxy.

mkreth commented 2 years ago

@nareshahi have you seen https://github.com/logstash-plugins/logstash-input-salesforce/blob/main/docs/index.asciidoc#http-proxy which describes how to configure a proxy for the connection to Salesforce? Does this solve your problem?