search

logstash-plugins / logstash-filter-metrics

Apache License 2.0
15 stars 29 forks source link

Windows test failure #6

Closed jordansissel closed 9 years ago

jordansissel commented 9 years ago

From https://github.com/elasticsearch/logstash/issues/2487

  71) LogStash::Filters::Metrics with multiple instances counts should be independent
     Failure/Error: events_tag1 = filter_tag1.flush
     RuntimeError:
       expected UTF-8 encoding for value=sadness, encoding=#<Encoding:Windows-1252>
     # ./lib/logstash/event.rb:299:in `validate_value'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:57:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:112:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:170:in `run'

  72) LogStash::Filters::Metrics when a custom clear_interval is set should clear the metrics after interval has passed
     Failure/Error: insist { filter.flush.first["http.200.count"] } == 1 # 5s
     RuntimeError:
       expected UTF-8 encoding for value=sadness, encoding=#<Encoding:Windows-1252>
     # ./lib/logstash/event.rb:299:in `validate_value'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:57:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:112:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:170:in `run'

  73) LogStash::Filters::Metrics when custom rates and percentiles are selected on the first flush should include only the requested rates
     Failure/Error: filter.flush
     RuntimeError:
       expected UTF-8 encoding for value=sadness, encoding=#<Encoding:Windows-1252>
     # ./lib/logstash/event.rb:299:in `validate_value'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:57:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:112:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:170:in `run'

  74) LogStash::Filters::Metrics when custom rates and percentiles are selected on the first flush should include only the requested percentiles
     Failure/Error: filter.flush
     RuntimeError:
       expected UTF-8 encoding for value=sadness, encoding=#<Encoding:Windows-1252>
     # ./lib/logstash/event.rb:299:in `validate_value'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:57:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:112:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:170:in `run'

  75) LogStash::Filters::Metrics when custom rates and percentiles are selected on the first flush should flush counts
     Failure/Error: filter.flush
     RuntimeError:
       expected UTF-8 encoding for value=sadness, encoding=#<Encoding:Windows-1252>
     # ./lib/logstash/event.rb:299:in `validate_value'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:57:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:112:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:170:in `run'

  76) LogStash::Filters::Metrics when a custom flush_interval is set should flush only when required
     Failure/Error: insist { filter.flush.length } == 1 # 15s
     RuntimeError:
       expected UTF-8 encoding for value=sadness, encoding=#<Encoding:Windows-1252>
     # ./lib/logstash/event.rb:299:in `validate_value'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:57:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:112:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:170:in `run'

  77) LogStash::Filters::Metrics with timer config on the first flush should include rates and percentiles keys
     Failure/Error: filter.flush
     RuntimeError:
       expected UTF-8 encoding for value=sadness, encoding=#<Encoding:Windows-1252>
     # ./lib/logstash/event.rb:299:in `validate_value'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:57:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:112:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:170:in `run'

  78) LogStash::Filters::Metrics with timer config on the first flush should include max value
     Failure/Error: filter.flush
     RuntimeError:
       expected UTF-8 encoding for value=sadness, encoding=#<Encoding:Windows-1252>
     # ./lib/logstash/event.rb:299:in `validate_value'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:57:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:112:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:170:in `run'

  79) LogStash::Filters::Metrics with timer config on the first flush should include mean value
     Failure/Error: filter.flush
     RuntimeError:
       expected UTF-8 encoding for value=sadness, encoding=#<Encoding:Windows-1252>
     # ./lib/logstash/event.rb:299:in `validate_value'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:57:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:112:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:170:in `run'

  80) LogStash::Filters::Metrics with timer config on the first flush should include stddev value
     Failure/Error: filter.flush
     RuntimeError:
       expected UTF-8 encoding for value=sadness, encoding=#<Encoding:Windows-1252>
     # ./lib/logstash/event.rb:299:in `validate_value'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:57:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:112:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:170:in `run'

  81) LogStash::Filters::Metrics with timer config on the first flush should include min value
     Failure/Error: filter.flush
     RuntimeError:
       expected UTF-8 encoding for value=sadness, encoding=#<Encoding:Windows-1252>
     # ./lib/logstash/event.rb:299:in `validate_value'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:57:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:112:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:170:in `run'

  82) LogStash::Filters::Metrics with timer config on the first flush should flush counts
     Failure/Error: filter.flush
     RuntimeError:
       expected UTF-8 encoding for value=sadness, encoding=#<Encoding:Windows-1252>
     # ./lib/logstash/event.rb:299:in `validate_value'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:57:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:112:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:170:in `run'

  83) LogStash::Filters::Metrics with timer config on the first flush should include percentile value
     Failure/Error: filter.flush
     RuntimeError:
       expected UTF-8 encoding for value=sadness, encoding=#<Encoding:Windows-1252>
     # ./lib/logstash/event.rb:299:in `validate_value'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:57:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:112:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:170:in `run'

  84) LogStash::Filters::Metrics with basic meter config when events are received on the first flush should flush counts
     Failure/Error: filter.flush
     RuntimeError:
       expected UTF-8 encoding for value=sadness, encoding=#<Encoding:Windows-1252>
     # ./lib/logstash/event.rb:299:in `validate_value'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:57:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:112:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:170:in `run'

  85) LogStash::Filters::Metrics with basic meter config when events are received on the first flush should include rates and percentiles
     Failure/Error: filter.flush
     RuntimeError:
       expected UTF-8 encoding for value=sadness, encoding=#<Encoding:Windows-1252>
     # ./lib/logstash/event.rb:299:in `validate_value'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:57:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:112:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:170:in `run'

  86) LogStash::Filters::Metrics with basic meter config when events are received on the second flush should not reset counts
     Failure/Error: events = filter.flush
     RuntimeError:
       expected UTF-8 encoding for value=sadness, encoding=#<Encoding:Windows-1252>
     # ./lib/logstash/event.rb:299:in `validate_value'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:57:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:112:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:170:in `run'

  87) LogStash::Filters::Metrics with basic meter config when custom rates and percentiles are selected on the first flush should include only the requested rates
     Failure/Error: filter.flushq
     RuntimeError:
       expected UTF-8 encoding for value=sadness, encoding=#<Encoding:Windows-1252>
     # ./lib/logstash/event.rb:299:in `validate_value'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:57:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:112:in `run'
     # C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:170:in `run'
jsvd commented 9 years ago

On windows, encoding isn't utf-8, so the tests break:

irb(main):005:0> LogStash::Event.validate_value "hello"
RuntimeError: expected UTF-8 encoding for value=hello, encoding=#<Encoding:Windows-1252>
        from C:/lswin/logstash-cmdexe/lib/logstash/event.rb:299:in `validate_value'

This is called from logstash-devutils in the rspec_helpers we use:

class LogStash::Event
  alias_method :setval, :[]=
  def []=(str, value)
    if str == TIMESTAMP && !value.is_a?(LogStash::Timestamp)
      raise TypeError, "The field '@timestamp' must be a LogStash::Timestamp, not a #{value.class} (#{value})"
    end
    LogStash::Event.validate_value(value)
    setval(str, value)
  end # def []=
end

Commenting the validate_value makes tests pass. Otherwise the tests can be changed to force the input strings to be utf-8

jordansissel commented 9 years ago

The encoding should always be UTF-8 within logstash

jordansissel commented 9 years ago 
expected UTF-8 encoding for value=sadness

Ahh. 'sadness' is my hostname. Socket.gethostname returns strings in host encoding. There's a jruby bug I filed about this somewhere.

jsvd commented 9 years ago

sadness ensues

ph commented 9 years ago

fixed in a monkeypatch in https://github.com/elasticsearch/logstash/pull/2507