allinurl / goaccess

GoAccess is a real-time web log analyzer and interactive viewer that runs in a terminal in *nix systems or through your browser.
https://goaccess.io
MIT License
18.12k stars 1.1k forks source link

X-Forwarded-For field not parsing #2148

Closed minusf closed 3 years ago

minusf commented 3 years ago

goaccess 1.5.1 macos 11.4

I am not able to convince goaccess about the X-Forwarded-For field in my logs. There is nothing special about it:

test.log:
x.x.x.x - - [15/Jul/2021:14:18:11 +0200] "GET / HTTP/1.1" 200 33900 "-" "-" "y.y.y.y.y" 0.137 TLSv1.2
goaccess --log-format='%h %^[%d:%t %^] "%r" %s %b "%R" "%u" ~h{," } %T %^' test.log

This always shows x.x.x.x in the "Visitor Hostnames and IPs" (not y.y.y.y)

allinurl commented 3 years ago

For the given log, e.g.,

192.168.0.1 - - [15/Jul/2021:14:18:11 +0200] "GET / HTTP/1.1" 200 33900 "-" "-" "190.12.12.12" 0.137 TLSv1.2
192.168.0.1 - - [15/Jul/2021:14:18:11 +0200] "GET / HTTP/1.1" 200 33900 "-" "-" "10.12.12.12, 21.2.212.2" 0.137 TLSv1.2

This should work:

goaccess access.log --log-format='%^[%d:%t %^] "%r" %s %b "%R" "%u" ~h{," } %T %K' --date-format=%d/%b/%Y --time-format=%T
minusf commented 3 years ago

so %h and ~h are exclusive? is this mentioned somewhere in the man page?

allinurl commented 3 years ago

Part of it was in the man page, under the specifiers section. I expanded this to hosts as well.

Important If multiple time served specifiers or hosts are used at the same time, the first option specified in the format string will take priority over the other specifiers.

minusf commented 3 years ago

ok, that helps. but i am still confused becasue %h != ~h, one is host, the other is x-forwarded-for, they might be both just ip numbers but they are specifying 2 entirely different things.. i see that %x is already taken but maybe ~X would be a less confusing specifier for it?

allinurl commented 3 years ago

Good point, the reason it uses ~h is that whatever is extracted either through ~h or %h, that's what it's going to be put in the hosts panel.

minusf commented 3 years ago

ok, that also kind of makes sense :} if some of this could be captured in the man page, i think that would be great! i'll close this now if it's fine with you. thank you for the help.