Data type conversion and compatiblity with Logstash's grok

[jweygandt]

Is there a desire to have some compatibility with Logstash's grock? The separator character for logstash is a ":" (http://logstash.net/docs/1.4.1/filters/grok).

FYI - I think this is easy: Converter, line 44: String[] spec = key.split(";:"); // notice the addition of ":"

And to ensure you have good tests, in ApacheDataTypeTest, after line 45, add: assertFalse(json.contains("grokfailure"));

[kroepke]

I actually think this would break the date format part, because that can easily contain ':' characters.

[jweygandt]

Let's try it again. I'm making some improvements in the test, as the "==null" and "instanceof" is not quite enough.

What about: String[] spec = key.split(":|;",3);

Or would we need something slightly more complex?

[jweygandt]

FYI - the new test in ApacheDataTypeTest

@Test public void test003_one_line() throws GrokException, IOException, ParseException { Grok g = Grok.create("patterns/patterns", "%{COMMONAPACHELOG_DATATYPED}");

  String line = "64.242.88.10 - - [07/Mar/2004:16:45:56 -0800] \"GET /twiki/bin/attach/Main/PostfixCommands HTTP/1.1\" 401 12846";

  System.out.println(line);
  Match gm = g.match(line);
  gm.captures();

  String json = gm.toJson();
  assertNotEquals("{\"Error\":\"Error\"}", json);
  assertFalse(json.contains("grokfailure"));

  Map<String, Object> map = gm.toMap();
  assertTrue(map.get("timestamp").equals(new SimpleDateFormat("dd/MMM/yyyy:HH:mm:ss Z").parse("07/Mar/2004:16:45:56 -0800")));
  assertTrue(map.get("response").equals(Integer.valueOf(401)));
  assertTrue(map.get("ident").equals(Boolean.FALSE));
  assertTrue(map.get("httpversion").equals(Float.valueOf(1.1f)));
  assertTrue(map.get("bytes").equals(Long.valueOf(12846)));
  assertTrue(map.get("verb").equals("GET"));

}

[palmerabollo]

+1, this would be very useful.

[jrask]

+1, I agree.

[palmerabollo]

@kroepke This applies to the grok pattern (example: %{WORD:number:int}), not to the value that you want to parse (that might contain ':' characters)

[palmerabollo]

@jweygandt Could you send a pull request with that change? Thanks.

[jweygandt]

Just coming back after a while, I notice the pull has been merged.