search

logstash-plugins / logstash-output-file

Apache License 2.0
23 stars 53 forks source link

SystemExit if mkdir fails (e.g. due to permissions), instead of using `filename_failure`. #74

Open kreiger opened 5 years ago

kreiger commented 5 years ago

Using config:

output {
  if [application] {
    file {
      path => "/opt/%{application}-logs/%{application}.logstash"
      filename_failure => "/opt/logs/%{application}.logstash"
    }
  }
  file {
    path => "/opt/logs/logs.logstash"
  }
}

An event with "application":"logstash" arrives, but the /opt/logstash-logs/ directory does not exist, and can't be created due to insufficient permissions.

Expected behaviour: Logstash should fall back to filename_failure, which points to a directory that exists and is writeable by the logstash user.

Actual observed behaviour: Logstash shuts down.

[INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"6.4.2"}
...
[INFO ][logstash.outputs.file    ] Opening file {:path=>"/opt/logstash-logs/logstash.logstash"}
[INFO ][logstash.outputs.file    ] Creating directory {:directory=>"/opt/logstash-logs"}
[FATAL][logstash.runner          ] An unexpected error occurred! {:error=>#<Errno::EACCES: Permission denied - /opt/logstash-logs>, :backtrace=>["org/jruby/RubyDir.java:449:in `mkdir'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/fileutils.rb:253:in `fu_mkdir'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/fileutils.rb:227:in `block in mkdir_p'", "org/jruby/RubyArray.java:1818:in `reverse_each'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/fileutils.rb:225:in `block in mkdir_p'", "org/jruby/RubyArray.java:1734:in `each'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/fileutils.rb:211:in `mkdir_p'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-file-4.2.5/lib/logstash/outputs/file.rb:273:in `open'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-file-4.2.5/lib/logstash/outputs/file.rb:136:in `block in multi_receive_encoded'", "org/jruby/RubyHash.java:1343:in `each'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-file-4.2.5/lib/logstash/outputs/file.rb:135:in `block in multi_receive_encoded'", "org/jruby/ext/thread/Mutex.java:148:in `synchronize'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-file-4.2.5/lib/logstash/outputs/file.rb:134:in `multi_receive_encoded'", "/usr/share/logstash/logstash-core/lib/logstash/outputs/base.rb:87:in `multi_receive'", "org/logstash/config/ir/compiler/OutputStrategyExt.java:114:in `multi_receive'", "org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:97:in `multi_receive'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:373:in `block in output_batch'", "org/jruby/RubyHash.java:1343:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:372:in `output_batch'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:324:in `worker_loop'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:286:in `block in start_workers'"]}
[ERROR][org.logstash.Logstash    ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit