Stop lumberjack messages from different sources getting merged

[mrdavidlaing]

Using the multiline codec within the lumberjack input seems to

a) Cause crashes - see #353

{:timestamp=>"2014-03-02T08:13:19.130000+0000", :message=>"Using milestone 2 output plugin 'redis'. This plugin should be stable, but if you see strange behavior, please let us know! For more information on plugin milestones, see http://logstash.net/docs/1.3.2/plugin-milestones", :level=>:warn}
Exception in thread "LogStash::Runner" org.jruby.exceptions.RaiseException: (ArgumentError) invalid byte sequence in UTF-8
    at org.jruby.RubyRegexp.match(org/jruby/RubyRegexp.java:1699)
    at RUBY.match(file:/app/vendor/logstash.jar!/grok-pure.rb:144)
    at RUBY.decode(file:/app/vendor/logstash.jar!/logstash/codecs/multiline.rb:148)
    at RUBY.run(file:/app/vendor/logstash.jar!/logstash/inputs/lumberjack.rb:46)
    at org.jruby.RubyProc.call(org/jruby/RubyProc.java:271)
    at RUBY.data(file:/app/vendor/logstash.jar!/lumberjack/server.rb:241)
    at RUBY.run(file:/app/vendor/logstash.jar!/lumberjack/server.rb:223)
    at Lumberjack::Parser.data_field_value(file:/app/vendor/logstash.jar!/lumberjack/server.rb:182)
    at Lumberjack::Parser.feed(file:/app/vendor/logstash.jar!/lumberjack/server.rb:93)
    at RUBY.compressed_payload(file:/app/vendor/logstash.jar!/lumberjack/server.rb:198)
    at Lumberjack::Parser.feed(file:/app/vendor/logstash.jar!/lumberjack/server.rb:93)
    at RUBY.run(file:/app/vendor/logstash.jar!/lumberjack/server.rb:220)
    at RUBY.run(file:/app/vendor/logstash.jar!/lumberjack/server.rb:59)
rake aborted!

b) Merge values from different sources (possibly due to the crashes?) - see id: Et8oODUtS2mM6kvBAawwzw

    {"@timestamp":"2014-02-28T21:34:11.708Z","Latency_General_StaticPage":0.0038642,"@source.name":"ci-london-lon-ws01186","@source.lonlat":[1.23,0.123],"logger":"Monitors.StaticPageMonitor","level":"INFO"}
true^^false^^^^localhost^^^true^^^openwire^^^^^^^2014-02-27T21:45:16.7565505+00:00^^^^^^^^^^^^^^^^^^^^^^^^^^^0^^0^^^/172.16.66.61_55036^^^^^^^^^^^^^^/172.16.66.61:55036^^^^false^^^^^^^^^^^^^^^^^^^^^Connection^^^address

This needs to be fixed

[mrdavidlaing]

This has been running on logsearch-live since 2014-03-02T08:29:59.433000+0000, and appears to resolve both issues

[mrdavidlaing]

Running in live for the last few days, this:

(a) definitely stops the crashes (b) Seems to resolve the message merging problem (although this is hard to detect); I've been searching for tags:("_csvparsefailure"