Add the ability to set the timeout for Ruby's NET timeout

[Mauro-Rizzi]

So i'm currently running into the following error:

Error: Net::ReadTimeout Exception: Faraday::TimeoutError Stack: uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/protocol.rb:181:inrbuf_fill' uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/protocol.rb:157:in readuntil' uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/protocol.rb:167:inreadline' uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http/response.rb:40:in read_status_line' uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http/response.rb:29:inread_new' uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:1504:in block in transport_request' org/jruby/RubyKernel.java:1189:incatch' uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:1501:in transport_request' uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:1474:inrequest' uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:1467:in block in request' uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:914:instart' uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:1465:in request' uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:1223:inget' redacted/vendor/bundle/jruby/2.5.0/gems/faraday-0.15.4/lib/faraday/adapter/net_http.rb:85:in perform_request' redacted/vendor/bundle/jruby/2.5.0/gems/faraday-0.15.4/lib/faraday/adapter/net_http.rb:43:inblock in call' redacted/vendor/bundle/jruby/2.5.0/gems/faraday-0.15.4/lib/faraday/adapter/net_http.rb:92:in with_net_http_connection' redacted/vendor/bundle/jruby/2.5.0/gems/faraday-0.15.4/lib/faraday/adapter/net_http.rb:38:incall' redacted/vendor/bundle/jruby/2.5.0/gems/restforce-4.3.0/lib/restforce/middleware/custom_headers.rb:11:in call' redacted/vendor/bundle/jruby/2.5.0/gems/restforce-4.3.0/lib/restforce/middleware/gzip.rb:14:incall' redacted/vendor/bundle/jruby/2.5.0/gems/faraday_middleware-0.14.0/lib/faraday_middleware/response_middleware.rb:31:in call' redacted/vendor/bundle/jruby/2.5.0/gems/faraday-0.15.4/lib/faraday/response.rb:8:incall' redacted/vendor/bundle/jruby/2.5.0/gems/faraday_middleware-0.14.0/lib/faraday_middleware/response/follow_redirects.rb:87:in perform_with_redirection' redacted/vendor/bundle/jruby/2.5.0/gems/faraday_middleware-0.14.0/lib/faraday_middleware/response/follow_redirects.rb:75:incall' redacted/vendor/bundle/jruby/2.5.0/gems/restforce-4.3.0/lib/restforce/middleware/instance_url.rb:13:in call' redacted/vendor/bundle/jruby/2.5.0/gems/restforce-4.3.0/lib/restforce/middleware/authorization.rb:11:incall' redacted/vendor/bundle/jruby/2.5.0/gems/restforce-4.3.0/lib/restforce/middleware/authentication.rb:16:in call' redacted/vendor/bundle/jruby/2.5.0/gems/faraday_middleware-0.14.0/lib/faraday_middleware/request/encode_json.rb:24:incall' redacted/vendor/bundle/jruby/2.5.0/gems/restforce-4.3.0/lib/restforce/middleware/multipart.rb:16:in call' redacted/vendor/bundle/jruby/2.5.0/gems/restforce-4.3.0/lib/restforce/middleware/mashify.rb:8:incall' redacted/vendor/bundle/jruby/2.5.0/gems/faraday-0.15.4/lib/faraday/rack_builder.rb:143:in build_response' redacted/vendor/bundle/jruby/2.5.0/gems/faraday-0.15.4/lib/faraday/connection.rb:387:inrun_request' redacted/vendor/bundle/jruby/2.5.0/gems/faraday-0.15.4/lib/faraday/connection.rb:138:in get' redacted/vendor/bundle/jruby/2.5.0/gems/restforce-4.3.0/lib/restforce/concerns/verbs.rb:37:inblock in define_verb' redacted/vendor/bundle/jruby/2.5.0/gems/restforce-4.3.0/lib/restforce/concerns/verbs.rb:63:in block in define_api_verb' redacted/vendor/bundle/jruby/2.5.0/gems/restforce-4.3.0/lib/restforce/concerns/api.rb:186:inquery' redacted/vendor/bundle/jruby/2.5.0/gems/logstash-input-salesforce-3.0.7/lib/logstash/inputs/salesforce.rb:97:in run' redacted/logstash-core/lib/logstash/java_pipeline.rb:405:ininputworker' redacted/logstash-core/lib/logstash/java_pipeline.rb:396:in block in start_input'

While trying to ingest a week's worth of data in a single run.

This seems to happen because the plugin is taking too long to read salesforce's response

I might be mistaken, but if that's really the reason I would like a sfdc_timeout setting to be added to the plugin so we can override it and let it run as long as we want, especially considering there's no proper self recurrence in the plugin and also no way to know what was the last thing you queried.

[Mauro-Rizzi]

I let the plugin restart in a loop for a while trying to figure out this issue and suddenly it was able to process a week's worth of data without timing out. It seems my assumption might have not been on point but this still warrants testing in order to find the cause.

[Asafkbalink]

@mkreth This is needed for heavier queries. Is there any option to do that? Or, is there an alternative? (3.5m records, will run once a month~)

[Asafkbalink]

Added a pull request to support it, even though Salesforce forces the timeout for timeout which is bigger than 2 minutes~

32