search

allinurl / goaccess

GoAccess is a real-time web log analyzer and interactive viewer that runs in a terminal in *nix systems or through your browser.
https://goaccess.io
MIT License
18.12k stars 1.1k forks source link

Unable to parse Nginx logs in JSON format #2127

Closed eeraser710 closed 3 years ago

eeraser710 commented 3 years ago

Logs format :- log_format main escape=json '{' '"Request_Logged_Time":"$time_local",' '"Client_PUB_IPs":"$http_x_forwarded_for",' '"Remote_addr":"$remote_addr"' '}';

Sample logs:- {"Request_Logged_Time":"31/May/2021:03:41:11 +1000","Client_PUB_IPs":"","Remote_addr":"10.9.96.68"} {"Request_Logged_Time":"31/May/2021:03:41:11 +1000","Client_PUB_IPs":"","Remote_addr":"10.9.96.68"} {"Request_Logged_Time":"31/May/2021:03:41:11 +1000","Client_PUB_IPs":"","Remote_addr":"10.9.96.68"} {"Request_Logged_Time":"31/May/2021:03:41:11 +1000","Client_PUB_IPs":"","Remote_addr":"10.9.96.68"} {"Request_Logged_Time":"31/May/2021:03:41:11 +1000","Client_PUB_IPs":"","Remote_addr":"10.9.96.68"} {"Request_Logged_Time":"31/May/2021:03:41:11 +1000","Client_PUB_IPs":"","Remote_addr":"10.9.96.68"} {"Request_Logged_Time":"31/May/2021:03:41:11 +1000","Client_PUB_IPs":"","Remote_addr":"10.9.96.68"} {"Request_Logged_Time":"31/May/2021:03:41:11 +1000","Client_PUB_IPs":"","Remote_addr":"10.9.96.68"} {"Request_Logged_Time":"31/May/2021:03:41:11 +1000","Client_PUB_IPs":"","Remote_addr":"10.9.96.68"}

Command used:- goaccess --log-format='%^:"%d:%t %^",%^:%^,%^:"%h"%^' --date-format=%d/%b/%Y --time-format=%H:%M:%S -f sample

Error:- ==10003== GoAccess - Copyright (C) 2009-2020 by Gerardo Orellana ==10003== https://goaccess.io - hello@goaccess.io ==10003== Released under the MIT License. ==10003== ==10003== FILE: sample ==10003== Parsed 9 lines producing the following errors: ==10003== ==10003== A request is required. ==10003== A request is required. ==10003== A request is required. ==10003== A request is required. ==10003== A request is required. ==10003== A request is required. ==10003== A request is required. ==10003== A request is required. ==10003== A request is required. ==10003== ==10003== Format Errors - Verify your log/date/time format

Kindly help me to know whats wrong .

eeraser710 commented 3 years ago

Tried below command ,,, but still same error

goaccess --log-format='{"Request_Logged_Time":"%d:%t %^","Client_PUB_IPs":"%^","Remote_addr":"%h"}' --date-format=%d/%b/%Y --time-format=%T -f sample

==10140== GoAccess - Copyright (C) 2009-2020 by Gerardo Orellana ==10140== https://goaccess.io - hello@goaccess.io ==10140== Released under the MIT License. ==10140== ==10140== FILE: sample ==10140== Parsed 9 lines producing the following errors: ==10140== ==10140== A request is required. ==10140== A request is required. ==10140== A request is required. ==10140== A request is required. ==10140== A request is required. ==10140== A request is required. ==10140== A request is required. ==10140== A request is required. ==10140== A request is required. ==10140== ==10140== Format Errors - Verify your log/date/time format

eeraser710 commented 3 years ago

Got it working .. was missing request in logs , thus goaccess was unable to parse it ...

GoAccess requires the following fields:

a valid IPv4/6 %h a valid date %d the request %r