search

logstash-plugins / logstash-filter-translate

Translate filter for Logstash
Apache License 2.0
21 stars 47 forks source link

psych 5.1.1 causing logstash to crash with logstash-filter-translate@3.4.2 #100

Closed mevatron closed 1 year ago

mevatron commented 1 year ago

Logstash information:

Please include the following information:

  1. Logstash version (e.g. bin/logstash --version) 8.6.2
  2. Logstash installation source (e.g. built from source, with a package manager: DEB/RPM, expanded from tar or zip archive, docker) Debian package direct from Elastic
  3. How is Logstash being run (e.g. as a service/service manager: systemd, upstart, etc. Via command line, docker/kubernetes) Docker
  4. How was the Logstash Plugin installed
bin/logstash-plugin update logstash-filter-translate

JVM (e.g. java -version):

If the affected version of Logstash is 7.9 (or earlier), or if it is NOT using the bundled JDK or using the 'no-jdk' version in 7.10 (or higher), please provide the following information:

  1. JVM version (java -version)
  2. JVM installation source (e.g. from the Operating System's package manager, from source, etc).
  3. Value of the JAVA_HOME environment variable if set.

OS version (uname -a if on a Unix-like system): Docker ubuntu:22.04

Description of the problem including expected versus actual behavior: Prior to the psych 5.1.1 release on 5.1.0 everything worked. Now the pipeline fails with the below stacktrace. Steps to reproduce:

Please include a minimal but complete recreation of the problem, including (e.g.) pipeline definition(s), settings, locale, etc. The easier you make for us to reproduce it, the more likely that somebody will take the time to look at it.

  1. I believe pinning psych to 5.1.0 should fix the issue for now

Provide logs (if relevant):

[ERROR] 2023-10-12 00:40:17.840 [[s3_output]-pipeline-manager] javapipeline - Pipeline error {:pipeline_id=>"s3_output", :exception=>#<NoMethodError: undefined method `code_point_limit=' for #<Psych::Parser:0x32a26bbb>>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-filter-translate-3.4.2/lib/logstash/filters/dictionary/yaml_file.rb:14:in `initialize_for_file_type'", "/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-filter-translate-3.4.2/lib/logstash/filters/dictionary/file.rb:42:in `initialize'", "org/jruby/RubyClass.java:911:in `new'", "/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-filter-translate-3.4.2/lib/logstash/filters/dictionary/file.rb:14:in `create'", "/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-filter-translate-3.4.2/lib/logstash/filters/translate.rb:198:in `register'", "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:75:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:234:in `block in register_plugins'", "org/jruby/RubyArray.java:1865:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:233:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:601:in `maybe_setup_out_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:246:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:191:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:143:in `block in start'"], "pipeline.sources"=>["/etc/logstash/s3_output.conf"], :thread=>"#<Thread:0x40d07478@/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:131 run>"}
[INFO ] 2023-10-12 00:40:17.842 [[s3_output]-pipeline-manager] javapipeline - Pipeline terminated {"pipeline.id"=>"s3_output"}
[ERROR] 2023-10-12 00:40:17.851 [Converge PipelineAction::Create<s3_output>] agent - Failed to execute action {:id=>:s3_output, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<s3_output>, action_result: false", :backtrace=>nil}
mevatron commented 1 year ago

Looks like psych corrected the issue upstream: https://github.com/ruby/psych/issues/655