allinurl / goaccess

GoAccess is a real-time web log analyzer and interactive viewer that runs in a terminal in *nix systems or through your browser.
https://goaccess.io
MIT License
17.77k stars 1.09k forks source link

Logging Format Question - HAProxy #2678

Open UpperCenter opened 1 month ago

UpperCenter commented 1 month ago

Hi allinurl,

Thanks for the fantastic tool, I have a few logging queries and issues relating to HAProxy. I've configured my setup to use clf log formatting, which I presumed would work out of the box with Goaccess as it supports. I see this error when using the below setup:

Token '2024-05-07T00:00:16.574522+00:00' doesn't match specifier '%

Example log output:

2024-05-07T08:39:37.326492+00:00 ip-172-21-111-50 haproxy[22009]: 172.21.102.219 - - [07/May/2024:08:39:37 +0000] "POST /some/uri HTTP/1.1" 401 271 "" "" 25792 302 "ft~" "bk01" "srv01" 0 0 4 3 7 ---- 19 19 0 0 0 0 0 "" ""
2
log-format "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r"

I followed the official HAProxy CLF Docs to setup the log formatting.

What's the best log-format to take advantage of as many Goaccess features as possible with HAProxy? Should I be using Common Log Format (CLF) or Common Log Format (CLF) with Virtual Host?

Thanks again

allinurl commented 1 week ago

Apologies for the delay in seeing this post. The following should help, although I'm unsure about some of those additional fields. Please feel free to share them here, and I'll take a look.

# goaccess access.log --log-format='%^]: %h %^[%x] "%r" %s %b "%R" "%u" %^' --datetime-format='%d/%b/%Y:%H:%M:%S %z' --tz=America/New_York --date-spec=min