LFV doesn't work on Logstash 7.13

[matejzero]

Hey,

I tried running LFV on logstash 7.13 today and it failed.

It exited with errors:

2021/06/10 14:40:14 Error while accept unix socket: accept unix /tmp/341560671/socket: use of closed network connection
2021/06/10 14:40:14 Error while accept unix socket: accept unix /tmp/729866473/socket: use of closed network connection

Running with --logstash-output and --loglevel=DEBUG doesn't give any more info.

I tried running LFV and aborting it after it created tmp folders and than manually ran CLI command: /usr/share/logstash/bin/logstash -w 1 --debug -f /tmp/077523468/pipeline.d -b 1 -l /tmp/077523468/log --path.settings /tmp/077523468/config --path.data /tmp/077523468/data

I get an error about GROK patterns:

[2021-06-10T12:55:01,094][ERROR][logstash.javapipeline    ][main] Pipeline error {:pipeline_id=>"main", :exception=>#<Grok::PatternError: pattern %{POSTFIX_QMGR} not defined>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/jls-grok-0.11.5/lib/grok-pure.rb:123:in `block in compile'", "org/jruby/RubyKernel.java:1442:in `loop'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/jls-grok-0.11.5/lib/grok-pure.rb:93:in `compile'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-grok-4.4.0/lib/logstash/filters/grok.rb:282:in `block in register'", "org/jruby/RubyArray.java:1809:in `each'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-grok-4.4.0/lib/logstash/filters/grok.rb:276:in `block in register'", "org/jruby/RubyHash.java:1415:in `each'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-grok-4.4.0/lib/logstash/filters/grok.rb:271:in `register'", "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:75:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:228:in `block in register_plugins'", "org/jruby/RubyArray.java:1809:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:227:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:586:in `maybe_setup_out_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:240:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:185:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:137:in `block in start'"], "pipeline.sources"=>["/tmp/077523468/pipeline.d/11-filter-syslog-prefilter.conf", "/tmp/077523468/pipeline.d/20-filter-amavis.conf", "/tmp/077523468/pipeline.d/22-filter-dovecot.conf", "/tmp/077523468/pipeline.d/23-filter-postfix.conf", "/tmp/077523468/pipeline.d/26-filter-simplesaml.conf", "/tmp/077523468/pipeline.d/27-filter-radiusd.conf", "/tmp/077523468/pipeline.d/28-filter-dhcpd.conf", "/tmp/077523468/pipeline.d/30-filter-httpd-app.conf", "/tmp/077523468/pipeline.d/32-filter-ldap.conf", "/tmp/077523468/pipeline.d/33-filter-hw.conf", "/tmp/077523468/pipeline.d/34-filter-sshd.conf", "/tmp/077523468/pipeline.d/35-filter-temporary.conf", "/tmp/077523468/pipeline.d/36-filter-pex.conf", "/tmp/077523468/pipeline.d/37-filter-rhds.conf", "/tmp/077523468/pipeline.d/40-filter-nagios.conf", "/tmp/077523468/pipeline.d/45-filter-arc.conf", "/tmp/077523468/pipeline.d/46-filter-slurm.conf", "/tmp/077523468/pipeline.d/47-filter-ftp.conf", "/tmp/077523468/pipeline.d/50-filter-wowza.conf", "/tmp/077523468/pipeline.d/51-filter-gunicorn.conf", "/tmp/077523468/pipeline.d/51-filter-openvpn.conf", "/tmp/077523468/pipeline.d/52-filter-django.conf", "/tmp/077523468/pipeline.d/53-filter-tftpd.conf", "/tmp/077523468/pipeline.d/54-filter-wlc.conf", "/tmp/077523468/pipeline.d/55-filter-fail2ban.conf", "/tmp/077523468/pipeline.d/56-filter-puppet.conf", "/tmp/077523468/pipeline.d/57-filter-auditbeat.conf", "/tmp/077523468/pipeline.d/60-filter-tagging.conf", "/tmp/077523468/pipeline.d/69-filter-cleaning.conf", "/tmp/077523468/pipeline.d/ioconfig.975570427.conf"], :thread=>"#<Thread:0x44f33984 run>"}

[2021-06-10T12:55:01,126][ERROR][logstash.agent           ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}

I do have custom grok rules in patterns/ folder.

It work with LFV 7.12 and lower, but stops working with 7.13. I checked changelog for any obvious change, but couldn't find anything.

So far I don't have any more info, but will try and dig deeper.

[matejzero]

I tested 7.12.1 and 7.13.0 and first one is working and the second one is not.

[magnusbaeck]

Is this with LFV 1.6.3?

[matejzero]

Well, I should post that as part of an issue. Basic info😀

yes, that’s on 1.6.3.

[breml]

Just for the record, the integration test suite passed for me with the current master (LFV 2.0) and Logstash 7.13.0.

[matejzero]

The issue was with geoip filter: https://github.com/logstash-plugins/logstash-filter-geoip/pull/185

Logstash 7.13.2+ is needed for the fix.