benoitc / gunicorn

gunicorn 'Green Unicorn' is a WSGI HTTP Server for UNIX, fast clients and sleepy applications.
http://www.gunicorn.org
Other
9.81k stars 1.75k forks source link

syslog unsupports rfc3164 #3310

Open ezinall opened 2 weeks ago

ezinall commented 2 weeks ago

I tried to send logs to fluentd by syslog. Fleuntd shows me error: 2024-10-03 15:38:32 +0000 [error]: #0 invalid input data="<14>gunicorn.site.wsgi.application.access: [30] 10.0.0.2 - USER [03/Oct/2024:18:38:32 +0300] \"POST /api/blank/ HTTP/1.0\" 201 292 \"-\" \"Mozilla/5.0\"\x00" error_class=Fluent::TimeParser::TimeParseError error="invalid time format: value = gunicorn.site.wsgi.application.access: [30] 10.0.0.2, error_class = ArgumentError, error = string doesn't match" I checked format of message and i have found mistake:

There isn't time and should be gunicorn.site.wsgi.application.access[30]: i think.

There is solution for this problem? I suggest to add timestamp to syslog message and fix tag. Result: <14>Oct 03 18:38:32 gunicorn.site.wsgi.application.access[30]: 10.0.0.2 - USER [03/Oct/2024:18:38:32 +0300] "POST /api/blank/ HTTP/1.0" 201 292 "-" "Mozilla/5.0"

https://regex101.com/r/MhhzJ2/1

pajod commented 2 weeks ago

RFC 3164 describes the (ridiculous) syslog format, but access log mimics (also ridiculous) w3c/apache httpd conventions.

Set your own --access-logformat if the non-standard time format works as long as it comes first, or extend gunicorn.glogging.Logger if not.

ezinall commented 1 week ago

At first access_fmt(access-logformat) and syslog_fmt has different value. The problem is in the header of the syslog message format. syslog_fmt = "[%(process)d] %(message)s" => syslog_fmt = "[%(process)d]: %(message)s" https://github.com/benoitc/gunicorn/blob/master/gunicorn/glogging.py#L179 and fmt = logging.Formatter(r"%s: %s" % (prefix, fmt)) => fmt = logging.Formatter(r"%s%s" % (prefix, fmt)) https://github.com/benoitc/gunicorn/blob/master/gunicorn/glogging.py#L440 Colon in the wrong place!

At second the flag of syslog header time is needed.