Error registering plugin Logstash5.5.0 Debian9

[zhu-jw]

Moved from ES output failing with long message Logstash 5.5.0 Debian9 #631

• Version: Logstash5.5.0 Elasticsearch5.5.0

• Operating System:Debian9

• Config File:

  input {
  beats {
      port => "5043"
  }
  }
  filter {
  grok {
      match => { "message" => "%{COMBINEDAPACHELOG}"}
  }
  geoip {
      source => "clientip"
  }
  }
  output {
  elasticsearch {
      hosts => [ "localhost:9200" ]
  }
  }

• Steps to Reproduce:

  10:42:45.788 [Ruby-0-Thread-3: /usr/share/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/task.rb:22] WARN  logstash.agent - fetched new config for pipeline. upgrading.. {:pipeline=>"main", :config=>"input {\n\t\tbeats {\n\t\t\t\tport => \"5043\"\n\t\t}\n}\n# The filter part of this file is commented out to indicate that it is\n# optional.\nfilter {\n\t\tgrok {\n\t\t\t\tmatch => { \"message\" => \"%{COMBINEDAPACHELOG}\"}\n\t\t}\n\t\tgeoip {\n\t\t\t\tsource => \"clientip\"\n\t\t}\n}\noutput {\n\t\telasticsearch {\n\t\t\t\thosts => [\"http://192.168.220.123:9200\"]\n\t\t}\n}\n\n"}
  10:42:45.789 [Ruby-0-Thread-3: /usr/share/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/task.rb:22] WARN  logstash.agent - stopping pipeline {:id=>"main"}
  10:42:50.829 [Ruby-0-Thread-36: /usr/share/logstash/logstash-core/lib/logstash/shutdown_watcher.rb:31] WARN  logstash.shutdownwatcher - {"inflight_count"=>0, "stalling_thread_info"=>{"other"=>[{"thread_id"=>90, "name"=>"[main]<beats", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/logstash-input-beats-3.1.21-java/lib/logstash/inputs/beats.rb:206:in `run'"}], ["LogStash::Filters::GeoIP", {"source"=>"clientip", "id"=>"ab1892fc61a4dfa666c5a9ed87ba5d230a195067-3"}]=>[{"thread_id"=>80, "name"=>"[main]>worker0", "current_call"=>"[...]/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:147:in `lock'"}, {"thread_id"=>81, "name"=>"[main]>worker1", "current_call"=>"[...]/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:147:in `lock'"}]}}
  10:42:50.832 [Ruby-0-Thread-36: /usr/share/logstash/logstash-core/lib/logstash/shutdown_watcher.rb:31] ERROR logstash.shutdownwatcher - The shutdown process appears to be stalled due to busy or blocked plugins. Check the logs for more information.
  10:42:52.655 [[main]-pipeline-manager] ERROR logstash.pipeline - Error registering plugin {:plugin=>"#<LogStash::OutputDelegator:0x357fc5e7 @namespaced_metric=#<LogStash::Instrument::NamespacedMetric:0x24eda258 @metric=#<LogStash::Instrument::Metric:0x4cf2383b @collector=#<LogStash::Instrument::Collector:0x26c7a9bc @agent=nil, @metric_store=#<LogStash::Instrument::MetricStore:0x6d5c107e @store=#<Concurrent::Map:0x00000000a01a34 entries=4 default_proc=nil>, @structured_lookup_mutex=#<Mutex:0x246a80ef>, @fast_lookup=#<Concurrent::Map:0x00000000a01a38 entries=74 default_proc=nil>>>>, @namespace_name=[:stats, :pipelines, :main, :plugins, :outputs, :\"aa8f4e22cd0da4e8237f9364d12406477de7a398-4\"]>, @metric=#<LogStash::Instrument::NamespacedMetric:0x62958b7c @metric=#<LogStash::Instrument::Metric:0x4cf2383b @collector=#<LogStash::Instrument::Collector:0x26c7a9bc @agent=nil, @metric_store=#<LogStash::Instrument::MetricStore:0x6d5c107e @store=#<Concurrent::Map:0x00000000a01a34 entries=4 default_proc=nil>, @structured_lookup_mutex=#<Mutex:0x246a80ef>, @fast_lookup=#<Concurrent::Map:0x00000000a01a38 entries=74 default_proc=nil>>>>, @namespace_name=[:stats, :pipelines, :main, :plugins, :outputs]>, @logger=#<LogStash::Logging::Logger:0x1f5a305f @logger=#<Java::OrgApacheLoggingLog4jCore::Logger:0x45465840>>, @strategy=#<LogStash::OutputDelegatorStrategies::Shared:0xa38a802 @output=<LogStash::Outputs::ElasticSearch hosts=>[http://192.168.220.123:9200], id=>\"aa8f4e22cd0da4e8237f9364d12406477de7a398-4\", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>\"plain_57d55755-2ae9-4f65-90aa-9a7d47e6e60f\", enable_metric=>true, charset=>\"UTF-8\">, workers=>1, index=>\"logstash-%{+YYYY.MM.dd}\", manage_template=>true, template_name=>\"logstash\", template_overwrite=>false, idle_flush_time=>1, doc_as_upsert=>false, script_type=>\"inline\", script_lang=>\"painless\", script_var_name=>\"event\", scripted_upsert=>false, retry_initial_interval=>2, retry_max_interval=>64, retry_on_conflict=>1, action=>\"index\", ssl_certificate_verification=>true, sniffing=>false, sniffing_delay=>5, timeout=>60, pool_max=>1000, pool_max_per_route=>100, resurrect_delay=>5, validate_after_inactivity=>10000, http_compression=>false>>, @id=\"aa8f4e22cd0da4e8237f9364d12406477de7a398-4\", @metric_events=#<LogStash::Instrument::NamespacedMetric:0x58e813dd @metric=#<LogStash::Instrument::Metric:0x4cf2383b @collector=#<LogStash::Instrument::Collector:0x26c7a9bc @agent=nil, @metric_store=#<LogStash::Instrument::MetricStore:0x6d5c107e @store=#<Concurrent::Map:0x00000000a01a34 entries=4 default_proc=nil>, @structured_lookup_mutex=#<Mutex:0x246a80ef>, @fast_lookup=#<Concurrent::Map:0x00000000a01a38 entries=74 default_proc=nil>>>>, @namespace_name=[:stats, :pipelines, :main, :plugins, :outputs, :\"aa8f4e22cd0da4e8237f9364d12406477de7a398-4\", :events]>, @output_class=LogStash::Outputs::ElasticSearch>", :error=>"SSLConnectionSocketFactory not found in packages org.apache.http.client.methods, org.apache.http.client.entity, org.apache.http.client.config, org.apache.http.config, org.apache.http.conn.socket, org.apache.http.impl, org.apache.http.impl.client, org.apache.http.impl.conn, org.apache.http.impl.auth, org.apache.http.entity, org.apache.http.message, org.apache.http.params, org.apache.http.protocol, org.apache.http.auth, java.util.concurrent, org.apache.http.client.protocol, org.apache.http.conn.ssl, java.security.cert, java.security.spec, java.security, org.apache.http.client.utils; last error: cannot load Java class org.apache.http.client.utils.SSLConnectionSocketFactory"}

  I can use it in Ubuntu 16.04.3 The error will stop the pipeline,Please help, Thanks

[jsvd]

I tried using that config with 5.5.1 and seems to work well, can you try with it?

[zhu-jw]

@jsvd thanks, maybe my system problem,but i can't solve it

[yaauie]

@RBICMP your error is different than the one reported in this 4 year old issue, and indicates that you have not allocated sufficient heap space. I would advise allocating more and trying again. If you have further issues, please ask in the community forums.