elastic / logstash

Logstash - transport and process your logs, events, or other data
https://www.elastic.co/products/logstash
Other
14.17k stars 3.49k forks source link

proper location for LogStash::Environment.pattern_path #4928

Open colinsurprenant opened 8 years ago

colinsurprenant commented 8 years ago

At some point LogStash::Environment.pattern_path was located in the bootstrap code, because of path issues but this creates some problems for code that executes outside the context of logstash, like specs, which requires LogStash::Environment.pattern_path.

Some temporary cut&paste fixes have been made to fox this but we need to clean this up. I also think we need to revisit all the code surrounding this LogStash::Environment.pattern_path which seems weird at some places.

The problem is detailed in logstash-plugins/logstash-filter-grok#72 and also relates to logstash-plugins/logstash-input-syslog#25 logstash-plugins/logstash-patterns-core#137

pgporada commented 7 years ago

I just started encountering this on Logstash 5.2.1. I'm just trying to write rspec tests for filters and have no idea how to start troubleshooting this.

filter {
    grok {
        match => { "message" => "%{SYSLOGLINE}" }
        overwrite => [ "message" ]
    }
}
# encoding: utf-8
require 'logstash/devutils/rspec/spec_helper'
require 'logstash/plugin'
require 'logstash/filters/grok'

describe 'simple syslog line' do
  file = File.open('/test/conf/filter.conf', 'rb')
  config file.read
  file.close

  message = %(Mar 16 00:01:25 evita postfix/smtpd[1713]: connect from tcnksm.net[168.100.1.3])

  sample('@message' => message) do
    insist { subject['message'] }.should be 'connect from tcnksm.net[168.100.1.3]'
    insist { subject['program'] }.should be 'postfix/smtpd'
  end
end
----> Run rspec of filtering
--- jar coordinate com.fasterxml.jackson.core:jackson-databind already loaded with version 2.7.4 - omit version 2.7.3
--- jar coordinate com.fasterxml.jackson.core:jackson-annotations already loaded with version 2.7.0 - omit version 2.7.3
--- jar coordinate com.fasterxml.jackson.core:jackson-core already loaded with version 2.7.4 - omit version 2.7.3
--- jar coordinate com.fasterxml.jackson.core:jackson-databind already loaded with version 2.7.4 - omit version 2.7.3
--- jar coordinate com.fasterxml.jackson.core:jackson-core already loaded with version 2.7.4 - omit version 2.7.3
Sending Logstash's logs to  which is now configured via log4j2.properties
NoMethodError: undefined method `pattern_path' for LogStash::Environment:Module
             Grok at /opt/logstash/vendor/jruby/lib/ruby/gems/shared/gems/logstash-filter-grok-3.3.1/lib/logstash/filters/grok.rb:232
           (root) at /opt/logstash/vendor/jruby/lib/ruby/gems/shared/gems/logstash-filter-grok-3.3.1/lib/logstash/filters/grok.rb:139
          require at org/jruby/RubyKernel.java:1040
          require at /opt/logstash/vendor/jruby/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:128
          require at /opt/logstash/vendor/jruby/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:121
          require at /opt/logstash/vendor/jruby/lib/ruby/gems/shared/gems/polyglot-0.3.5/lib/polyglot.rb:65
           (root) at /test/spec/filter_spec.rb:1
             load at org/jruby/RubyKernel.java:1059
           (root) at /test/spec/filter_spec.rb:4
             each at org/jruby/RubyArray.java:1613
           (root) at /opt/logstash/vendor/jruby/lib/ruby/gems/shared/gems/rspec-core-3.5.4/lib/rspec/core/configuration.rb:1
  load_spec_files at /opt/logstash/vendor/jruby/lib/ruby/gems/shared/gems/rspec-core-3.5.4/lib/rspec/core/configuration.rb:1435
  load_spec_files at /opt/logstash/vendor/jruby/lib/ruby/gems/shared/gems/rspec-core-3.5.4/lib/rspec/core/configuration.rb:1433
            setup at /opt/logstash/vendor/jruby/lib/ruby/gems/shared/gems/rspec-core-3.5.4/lib/rspec/core/runner.rb:100
              run at /opt/logstash/vendor/jruby/lib/ruby/gems/shared/gems/rspec-core-3.5.4/lib/rspec/core/runner.rb:86
              run at /opt/logstash/vendor/jruby/lib/ruby/gems/shared/gems/rspec-core-3.5.4/lib/rspec/core/runner.rb:71
             load at org/jruby/RubyKernel.java:1059
           (root) at /opt/logstash/vendor/jruby/bin/rspec:23