robcowart / elastiflow

Network flow analytics (Netflow, sFlow and IPFIX) with the Elastic Stack
Other
2.49k stars 598 forks source link

Exception in inputworker {"exception"=>#<EOFError: End of file reached> #675

Closed wangxin688 closed 3 years ago

wangxin688 commented 3 years ago

hello, I met this issue with logstash.inputs.udp plugin. the verison of elastiflow is 4.0.0 the log as follow:

[2020-12-29T23:04:56,140][ERROR][logstash.inputs.udp      ][elastiflow][20d857d994b054790dea8b96ec5f30a2cfc1505d7866ad85f88345e30fc2c233]
 Exception in inputworker {"exception"=>#<EOFError: End of file reached>, "backtrace"=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/bindata-2.4.8/lib/bindata/io.rb:316:in `read'",
  "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/bindata-2.4.8/lib/bindata/io.rb:335:in `accumulate_big_endian_bits'",
   "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/bindata-2.4.8/lib/bindata/io.rb:324:in `read_big_endian_bits'", 
   "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/bindata-2.4.8/lib/bindata/io.rb:297:in `readbits'", "(eval):30:in `read_and_return_value'",
   "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/bindata-2.4.8/lib/bindata/base_primitive.rb:129:in `do_read'",
    "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/bindata-2.4.8/lib/bindata/struct.rb:139:in `block in do_read'", "org/jruby/RubyArray.java:1809:in `each'",
    "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/bindata-2.4.8/lib/bindata/struct.rb:139:in `do_read'",
     "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/bindata-2.4.8/lib/bindata/array.rb:330:in `block in do_read'", "org/jruby/RubyArray.java:1809:in `each'",
     "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/bindata-2.4.8/lib/bindata/array.rb:330:in `do_read'", 
     "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/bindata-2.4.8/lib/bindata/struct.rb:139:in `block in do_read'", "org/jruby/RubyArray.java:1809:in `each'", 
     "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/bindata-2.4.8/lib/bindata/struct.rb:139:in `do_read'", "(eval):2:in `do_read'",
      "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/bindata-2.4.8/lib/bindata/struct.rb:139:in `block in do_read'", "org/jruby/RubyArray.java:1809:in `each'",
      "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/bindata-2.4.8/lib/bindata/struct.rb:139:in `do_read'", 
      "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/bindata-2.4.8/lib/bindata/array.rb:330:in `block in do_read'", "org/jruby/RubyArray.java:1809:in `each'", 
      "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/bindata-2.4.8/lib/bindata/array.rb:330:in `do_read'",
       "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/bindata-2.4.8/lib/bindata/struct.rb:139:in `block in do_read'", "org/jruby/RubyArray.java:1809:in `each'",
        "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/bindata-2.4.8/lib/bindata/struct.rb:139:in `do_read'",
         "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/bindata-2.4.8/lib/bindata/base.rb:147:in `block in read'", 
         "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/bindata-2.4.8/lib/bindata/base.rb:253:in `start_read'", 
         "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/bindata-2.4.8/lib/bindata/base.rb:145:in `read'", 
         "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/bindata-2.4.8/lib/bindata/base.rb:21:in `read'",
          "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-codec-sflow-2.0.0/lib/logstash/codecs/sflow.rb:105:in `decode'", 
          "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-udp-3.3.4/lib/logstash/inputs/udp.rb:151:in `inputworker'",
           "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-udp-3.3.4/lib/logstash/inputs/udp.rb:63:in `block in run'"]}
robcowart commented 3 years ago

What version of Logstash are you using? There was a bug a few releases back that would cause this problem.

wangxin688 commented 3 years ago

root@10923984-ESFLOW01:/home/admin# /usr/share/logstash/bin/logstash --version Using bundled JDK: /usr/share/logstash/jdk logstash 7.10.1 it's the latest version

wangxin688 commented 3 years ago

shoud I downgrade the version? And if need,do you have recommended version suggestion? thanks @

wangxin688 commented 3 years ago

What version of Logstash are you using? There was a bug a few releases back that would cause this problem. my logstash version is 7.10.1 and if I install logstash-codec-sflow directly, the version of this plugin is logstash-codec-sflow (2.0.0). And then I remove this and try to local install sflow plugin with version 2.13, reported this issue: root@10923984-ESFLOW01:/usr/share/logstash# bin/logstash-plugin install /tmp/logstash-codec-sflow-2.1.3/logstash-codec-sflow-2.1.3.gem

Validating /tmp/logstash-codec-sflow-2.1.3/logstash-codec-sflow-2.1.3.gem Installing logstash-codec-sflow Plugin version conflict, aborting ERROR: Installation Aborted, message: Bundler could not find compatible versions for gem "logstash-core": In snapshot (Gemfile.lock): logstash-core (= 7.10.1)

In Gemfile: logstash-core java

logstash-codec-sflow (= 2.1.3) java was resolved to 2.1.3, which depends on
  logstash-core (>= 5.4.0, <= 7.9.9) java

logstash-core-plugin-api java was resolved to 2.1.16, which depends on
  logstash-core (= 7.10.1) java

logstash-input-syslog java was resolved to 3.4.4, which depends on
  logstash-filter-grok java was resolved to 4.3.0, which depends on
    logstash-core (>= 5.6.0) java

logstash-integration-jdbc java was resolved to 5.0.6, which depends on
  logstash-core (>= 6.5.0) java

logstash-output-elasticsearch (>= 10.4.2) java was resolved to 10.7.0, which depends on
  logstash-mixin-ecs_compatibility_support (~> 1.0) java was resolved to 1.0.0, which depends on
    logstash-core (>= 5.0.0) java

Running bundle update will rebuild your snapshot from scratch, using only the gems in your Gemfile, which may resolve the conflict.

confused with this. @robcowart Realy upset after a lot investigation for this plugin

wangxin688 commented 3 years ago

finally, I reinstall logstash as verison 7.9.3 and fix this local install issue. close this.thanks