how about nginx error.log?

[zhourunlai]

e.g.

2016/05/04 18:14:15 [error] 3075#0: *11552 open() "/var/www/bak/mantoucc/favicon.ico" failed (2: No such file or directory), client: 115.175.32.29, server: mantoucc.com, request: "GET /favicon.ico HTTP/1.1", host: "mantoucc.com", referrer: "http://mantoucc.com/"

[jpoesen]

You can add your own named regex patterns to a parser object and then use them to construct the nginx error log format you need.

The following works for me on nginx/1.10.3 (Ubuntu), using a default nginx install, and nginx's default error log format:

$parser = new \Kassner\LogParser\LogParser();

$parser->addPattern('%NGXDT', '(?P<datetime>[\d+/ :]+)');
$parser->addPattern('%NGXLVL', '\[(?P<errorlevel>.+)\]');
$parser->addPattern('%NGXPID', '(?P<processid>\d+(?=\#))');
$parser->addPattern('%NGXTID', '(?P<threadid>(?<=\#)\d+)');
$parser->addPattern('%NGXCID', '(?P<connectionid>(?<=\:\s\*)\d+)');
$parser->addPattern('%NGXMSG', '(?P<message>.+)');
$parser->addPattern('%NGXCL', '(?P<client>.+)');
$parser->addPattern('%NGXSRV', '(?P<server>.+)');
$parser->addPattern('%NGXREQ', '(?P<request>.+)');
$parser->addPattern('%NGXUPS', '(?P<upstream>.+)');
$parser->addPattern('%NGXHST', '(?P<host>.+)');

$parser->setFormat('%NGXDT %NGXLVL %NGXPID\#%NGXTID: \*%NGXCID %NGXMSG, client: %NGXCL, server: %NGXSRV, request: "%NGXREQ", upstream: "%NGXUPS", host: "%NGXHST"');

NOTE:

• The above example is NOT widely tested, but works fine for me.
• You may need to add/remove patterns from the setFormat() call if your log lines contain different items.
• pay special attention to whitespace and non-alphanumeric characters. Any special characters you include in the format have to be escaped if they also have to be escaped when part of a 'standard' regex.

At any rate this should give a good start to anyone trying to parse nginx error logs with this parser.

It would be great if this (kind of) example could be added to the documentation. @kassner : I'd be happy to write up a page if you agree.

[kassner]

@jpoesen that is an awesome example on how use the library with custom logs, thanks for the effort!

Well, my intention with this library is parsing access log files only, but I don't think it would do any harm to keep this as an example for custom formats. I'll be happy to add it to the repository (maybe GitHub's wiki? Or a docs folder) with the content if you are willing to write it. Send a PR or ping me back with a Gist to put it on the Wiki if you want.

Thanks again @jpoesen!